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President's Message 



The 20th Annual Conference of the North East Association for Institutional Research, 
held November 6-9, 1993 at The Sagamore on Lake George in Bolton Landing, NY, was a 
tremendous success. This conference will undoubtedly be remembered by those who attended for 
its idyllic location and innovative program. 



This year's conference theme, "20 Years: Putting it All Together", was very 
appropriate and illustrative of our profession. What a long way our association has come in 
the twenty years since the inaugural conference was held in WiHiamstown, MA. The inaugural 
conference featured a keynote address, given by Lois Torrence, and three paper presentations. I 
think that it is quite fitting that at our 20th annual conference, the NEAIR membership voted 
to confer emeritus membership status upon Lois. Twenty years later our annual conference 
featured numerous workshops, paper presentations, panel discussions, special interest group 
meetings, and the inaugural team case study. 

Special thanks and credit are due to Marge Wiseman, Program Chair, and to Diane 
Cuneo, Local Arrangements Chair, for their excellent organization and planning. They set out 
to provide a conference that would take full advantage of the unique setting, to provide a rich 
learning environment and to foster collegial networking opportunities. The Sunday nighl 
keynote address by Robert Culver, Senior Vice President and Treasurer at Northeastern 
University, provided us with valuable insights and practical approaches to cost reductions and 
productivity enhancements. The team case study that was woven throughout the traditional 
parts of the program provided participants the opportunity to share and practice institutional 
research techniques. Many thanks go to Marian Pagano who conceived of and orchestrated this 
new conference component. The Monday evening dinner and Karaoke event was definitely an 
opportunity to make new friends, to reestablish old acquaintances and generally let our hair 
down. 



Congratulations to all the newly-elected NEAIR officers: to Marian Pagano on her 
election to the position of President-Elect, to Jane Price, on her election to Secretary, and to 
Darryl Bullock, Diane Cuneo, and Stuart Rich on their election to the Steering Committee. 
Additional thanks go to Jane Price who is serving double duty as Publications Chair and is 
responsible for the compilation and production of these Proceedings. The membership can be 
confident that these newly elected officers and steering committee members will serve the 
organization well. A special thank you to Larry Metzger and Tom Flaherty, who provided 
advice and guidance throughout my tenure as president. Heartfelt thanks go to Brenda Bretz, 
our membership secretary, for all her help and for the hours of service she provides this 
organization. Best wishes to Mike McGuire, a valued friend and colleague, as he assumes the 
responsibilities of the presidency. His new initiatives will strengthen and move NEAIR 
forward in the year ahead. With these acknowledgments, and heartfelt thanks to members of 
the Steering Committee and to the members of NEAiR, I commend to the membership these 
Proceedings. This has truly been a memorable year. 



Dawn Geronimo Terkla 
President, NEAIR 1992-93 
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Conference Program 



Saturday 



Newcomers to IR 

Michael F. Middaugh 

Director of IR & Planning 
University of Delaware 

Karen Bauer 
Senior Research Analyst 
University of Delaware 
Dale Trusheim 
Assoc. Director of IR & Planning 
University of Delaware 

1:30pm - 5:00pm 
Evelley 

Introduction to the 
Foundations of Total Quality 
Management and a Model of 
Continuous Improvement (Part I) 



Gregory Lozier 

Exec. Dir. of Planning and Analysis 
Pennsylvania State University 

Deborah J. Teeter 
Director of IR and Planning 
University of Kansas 
1:30pm - 5:00pm 
Abenia 



Statistics For IR, Session I 



Mary Ann Coughlin 

Assoc. Prof, of Research& Statistics 
1:30pm - 5:00pm 
Triuna A 



Understanding Campus 
and Politics 



Culture 



Dr. J. Frederipk^Volkwein 

UnivgiSity at Albany 

"1:30pm - 5:00pm 
Triuna B 



November 6 

This workshop is designed to give new practitioners in 
institutional research a hands-on approach to getting 
started in the field. Using the NEAIR Monograph "A 
Handbook for Newcomers to Institutional Research" 
workshop participants will walk through a series of 
exercises designed to address such issues as how to 
ensure data integrity; developing factbooks and reports 
that are used by college presidents; defining critical issues 
for institutional research at your college; identifying 
sources of data; conducting survey research; and devel- 
oping forecasting models. 

Workshop 

This workshop will aquaint participants with the concep- 
tual principles and foundations on which the pursuit of 
quality relies, as well as with some of the "tools" and 
methods of TQM. A problem-solving model will be used 
to illustrate how to apply these principles and tools. 
Working in teams is an aspect of TQM that will be both 
practiced and discussed. 

Workshop 



5:30pm - 7:00pm 



In Session I the very basic ideas in statifHcs will be 
covered in a way useful as an introduction or as a 
refresher to statistics. Descriptive statistics, sampling and 
probability theory as well as the inferential methods of chi 
square, t-test, and Pearson's r will be covered. 

Workshop 

This workshop acquaints institutional researchers with 
campus culture and politics, giving attention to both 
external environments and internal inner workings. The 
effectiveness of institutional research is enhanced when 
professionals understand the historical roots and orga- 
nizational dynamics of their institutions. Workshop par- 
ticipants will examine those organizational characteris- 
tics that makethese institutions more orlesseffective,and 
will discuss those campus features that have the greatest 
impact on the practice of institutional research. 

Workshop 

President's Reception (Sagamore Dining Room) 
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Sunday 

9:00am - 5:30pm 



Effective Presentations 
For Institutional Research 

Dale Trushehn 

Associate Director of 
Institutional Research 

and Planning 
University of Delaware 

9:00am » 12:00noon 
Evelley 



Total Quality Management 
(Session II) 

Gregory Lozier 

Executive Director 
Planning and Analysis 
Pennsylvania State University 

Deborah J. Teeter 

Director 
Institutional Research 

and Planning 
University of Kansas 

9:00am - 12:00noon 
Abenia 



Intermediate Statistics 
for Institutional Research 
(Session II) 



Mary Ann Coughlin 

Associate Professor 
of Research and Statistics 
Springfield College 

9:00am - 12:00noon 
Triuna / 



Beyond E-Mail: Getting 
Around On the Internet 

James Fergerson 
Director 
Institutional Research 
Bates College 

1:30pm - 5:00pm 
Evelley 



November 7 

Registration - Conference Center Lobby 

This workshop, aimed primarily at newcomers to 
Institutional Research, has two objectives. The first 
section of the workshop focuses on key principles of 
effective data presentation and reporting (design, use of 
color, tables vs. charts, types of charts, etc.). The sec- 
ond section will demonstrate relatively new computer 
technology which can also aid effective presentations. 
Computer getnerated slide shows, transparencies, and 
slides will be discussed and presented. 

Workshop 



This workshop will aquaint participants with the concep- 
tual principles and foundations on which the pursuit of 
quality relies, as well as with some of the ''tools" and 
methods of TQM. A problem-solving model will be used 
to illustrate how to apply these principles and tools. 
Working in teams is an aspect of TQM that will be both 
practiced and discussed. 

Workshop 



Session II will pick up with inferential statistics 
where Session I leaves off and will discuss such topics as 
Analysis of Variance, Regression, and Factor Analysis. 

Workshop 



The workshop will be an introduction to the variety of 
services available on the Internet. Topics covered will be: 
ftp, telnet, gopher, WAIS, finding information, data, and 
people on the networks, and news and discussion groups. 
(A basic understanding of electronic mail will be helpful). 

Workshop 
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NEATR Conference Program 



Choosing Technology 
for the IR Office 

Jennifer Wilton 

Director 
Office of Institutional Research 
jlJniversity of Massachusetts, Boston 

1:30pm - 5:00pm 
Abenia 



Understanding Campus 
Culture and Politic^ 
(Session llh 

J. Frederk)c£volkwein 

University at Albany 

1:30pm - 5:00pm 
Triuna A 



Catholic Colleges & Universities 



Peter Murray 

Catholic University of America 

4:00pm - 5:00pm 
Triuna B 



Mentor Prog ram Get-Together 

5:00pm - 5:30pm 
Triuna A 



Rightsizing - A Practical 
Approach to Cost Reduction and 
Productivity Enhancement 

Robert L. Culver 

Senior Vice President and Treasurer 
Northeastern University 

5:30pm - 6:30pm 
Conference Center - Bellvue 



6:30pm - 7:30pm 
7:30pm - 9:30pm 

9:30pm - 10:00pm 



The Sagamore, Lake George, 1993 



This workshop is designed to help those charged with 
technological decisions, particularly in new offices, to 
make the best choices possible for their situation. Factors 
covered will include the impact of the availability of and 
access to data, tools and technical support and the staffing 
skills of the office. 

Workshop 



This workshop acquaints institutional researchers with 
campus culture and politics, giving attention to both 
external environments and internal inner 
workings. The effectiveness of institutional research is 
enhanced when professionals understand the historical 
roots and organizational dynamics of their institutions. 
Workshop participants will examine those organizational 
characteristics that make these institutions more or less 
effective, and will discuss those campus features that have 
the greatest impact on the practice of institutional research. 

Workshop 

Representatives of Catholic colleges and universities are 
invited to share experiences and common concerns and to 
plan activities of mutual benefit. 

S/G 



General Session 



Reception & Cash Bar (Sagamore Dining Room) 
Banquet (Sagamore Dining Room) 

Initial Team Hudles (Sagamore Dining Room) 



3 13 



NEAIR Conference Program 



Monday 



The Sagamore, Lake George, 1993 



Monday 

6:30am - 7:30am 
8:00am - 11:00am 
7:30am - 8:00am 
7:45am - 8:45am 



Higher Education 
Data-Sharing 
Consortium (HEPS) 

James Trainer 
Director 
HEDS Consortium 
7:45am - 8:45am 
Evelley 



Two- Year Colleges 

Alan J. Sturtz 

Director 
Institutional Research 

South Central 
Community College 

7:45am - 8:45am 
Abenia A 



Penns^vania State System 
of Higher Education 

Richard Rugen 

Assistant to the president 
Planning & Institutional Technology 
Kutztown University 

7:45am - 8:45am 
Abenia B 



ASQ & ASQ+ Users Group 

Ellen Armstrong Kanarek 

Program Director 
Applied Educational Research, Inc. 

7:45am - 8:45am 
Triuna A 




Public Universities 
Information Exch 



Michael Mjddaugh 

Director ofJK&c Planning 
Uniyefsitv of Delaware 
7:45am - 8:45am 
Triuna B 




November 8 

Fun Run/Walk (Meet at Main Hotel Entrance) 
Registration (Conference Center Lobby) 
Breakfast (Conference Center - Nirvana) 
Special Interest Groups 

An opportunity for HEDS members and others interested 
in data exchange activities to discuss current plans and 
future areas of analysis, with a focus on institutional 
productivity and cost containment issues. 

SIG 



This SIG is intended for individuals dealing or concerned 
with the IR function in two-year institutions. Problems, 
concerns, and issues will be discussed in an informal 
setting. A representative of the National Council for 
Research and Planning will brief participants on NCRP 
plans for the coming year. 

SIG 



Institutional researchers from the State System in Pennsyl- 
vania will meet to discuss current issues and concerns 

SIG 



This session represents an opportunity for those interested 
in the Admitted Student Questionnaire or Admitted 
Student Questionnaire Plus to discuss their experiences, 
have their questions answered, and learn what changes 
may be planned. 

SIG 

An opportunity for members of the Exchange and others 
interested in data exchange activities in public institutions 
to discuss current plans, with a focus on studies of insti- 
tutional productivity, staffing, and cost reduction. 

SIG 
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Monday 
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The Sagamore, Lake George, 1993 



The Use of NCES 
National Databases for 
Institutional Analysis 

William H. Freund 

Chief 

Institutional Studies Branch 
National Center for Education 
Statistics 

Susan G. Broyles 

Section Head 
Institutional Studies, NCES 

Roslyn Korb 

Mathematical Statistician, NCES 

9:00am - 10:00am 
Evelley 



Designing Alumni Research 
for Assessment & Planning 

Anne Marie Delaney 

Director of Program Research 
Boston College 

9:00am - 9:45am 
Abenia 



Salary Equity: 
Comparison of Two 
Methodologies 

Denise A. Krallman 
Institutional Research Analyst 
Budgeting, Planning & Analysis 
Miami University 

9:00am - 9:45am 
Triuna A 



Key Demographic and 
Economic Factors Affecting 
Community Colleges 

Mar da M. Lee 

Director 
Office of Institutional 
Research & Planning 
Westchester Community College 

9:00am - 9:45am 
Triuna B 



This panel is designed to help participants gain hands-on 
experience in the use of several national databases that 
have rich information for institutional research. Thepanel 
will map major research issues that can be addressed with 
these databases, discuss technical issues such as sampling 
weights and variance computation in analyzing sample 
survey data, and illustrate and practice the use of PCs to 
create analysis files and conduct statistical analyses. Pro- 
cedures for obtaining these databases and technical as- 
sistance from NCES will also be discussed in the 
panel. 

Panel 

Moderator: John J. Casey, Georgian Court College 



The purpose of this paper is to present the design, imple- 
mentation strategies, analytical techniques and significant 
results from a recently completed undergraduate alumni 
study administered to School of Education graduating 
classes from 1987 through 1991. The paper will show how 
tfie methodology can be applied to different academic 
disciplines and professional schools, and how alumni 
research can be designed and utilized to meet various 
institutional research purposes including Accreditation, 
Assessment, Program Evaluation, and Program Planning. 

Paper 

Moderator: Amy Ensminger, Mansfield Univ. of PA 

Salary equity studies have traditionally utilized multiple 
regression as the method for identifying salary inequities. 
A new method of studying salary equity was introduced 
in 1991 . The purpose of this study was to use both methods 
in studying salary equity at Miami University and deter- 
mining consistency of their results. 

Paper 

Moderator: Phyliss A. Fitzpatrick, Fairfield University 



This paper seeks to identify the main economic and 
demographic factors affecting community colleges and 
discuss them in terms of planning needs for the future. 

Paper 

Moderator: Arthur Kramer, Passaic County 
Community College 
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Inferential Research Methods 
in Educational Aministration: 
Benefits and Limitations 

Caroline L. Pike 

Research Analyst 
Office of Institutional Research and 
Enrollment Planning 
Ithaca College 

9:55am - 10:35 
Abenia 



Electronically Accessible 
Survey Data Using Gopher 

Cynthia Lucia 

Staff Assistant/ 
Information Services 
Potsdam State College 

9:55am - 10:35am 
Triuna A 



Examples of How IR Can 
Help Campus Administrators 

Karen W. Bauer 

Senior Research Analyst 
Office of Institutional Research and 
Planning 
University of Delaware 

9:55am - 10:35am 
Triuna B 



10:35am - 10:45am 



10:45am - 12:15pm 



12:15pm - 1:45pm 



This paper will discuss how research methods may be 
applied to existing institutional data to address adminis- 
trative concerns. More specifically, the paper will describe 
data analysis techniques and how the results of inferential 
statistical methodologies can be used to inform decision 
making in the context of freshmen academic experience. 

Paper 

Moderator: John P. Jacobsen, State System of Higher Ed. 



At Potsdam College, we have begun to make survey 
response data accessible electronically to our campus 
community. Weuse client-server "freeware" called Gopher 
to "publish" the survey responses on the College-Wide- 
Information-System (CW1S). We encourage academic 
decision makers to review the narrative data and to make 
suggestions as to how we could revise it to present the 
College in the best possible light. 

Topical Case Study 
Moderator: Richard Kline, California Univ. of PA 

Many institutional researchers consult with campus 
colleagues in survey design and analysis. With three 
survey projects, the presenter will discuss the sequence of 
events (from initial contact with colleague requesting help 
through completion of a summary report) and will offer 
some of the "pleasures" and "nightmares" experienced in 
each project. To pi ca l Case Study 

Moderator: Daryl Bullock, Mercy College 



Break (Conference Center Lobby) 



Team Case Study Groups (Conference Center - Bellvue) 



Business Luncheon (Conference Center - Nirvana) 
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Developing a Survey 
to Assess the Graduate 
School Experience 

Jane Zeff 

Assistant Director 
Planning, Research & Evaluation 
William Patterson College 

2:00pm - 2:45pm 
Evelley 



An Analysis of Entering 
Freshmen Survey Data 
As It Relates to Retention 
of Students 



Linda LeFauve 

Research Associate 
Office of Institutional Analysis 
State University of New York 
at Buffalo 

Mark L. Molnar 

Senior Programmer 
Office of Institutional Analysis 
SUNY at Buffalo 

2:00pm - 2:45pm 
Abenia 



Making the Most of 
the Mission Review 

Eleanor Fujita 

Director 
Institutional Research 
Hudson Community College 

2:00pm - 2:45pm 
Triuna A 



Using ASQ to Determine 
Cognitive Fit Between 
Incoming Students and 
Future Enrollment 

Yun K. Kim 

Director 
Office of Institutional Research 
Goucher College 

2:00pm - 2:45pm 
Triuna B 



The Sagamore, Lake George, 1993 



Session attendees will review a survey being developed 
for the Graduate Office. Questions seek students' 
opinions about the quality of graduate services, library 
holdings, and the programs they have just completed. 
Other questions explore how students financed their 
graduate education and the reasons why they pursued an 
advanced degree. 

Topical Case Study 
Moderator: Denise A. Krallman, Miami University 



Freshman Survey data were analyzed to determine which 
variables were related to retention/graduation. Variables 
included were students' concern about financing their 
education, self-ratings of intellectual/social abilities, 
reasons for going to college and expectations of the college 
experience, and educational plans. Results of the analysis 
and programming issues will be discussed. 

Paper 

Moderator: Richard Rugen, Kutztown Univ. of PA 



A multi-faceted approach to the review of the college's 
mission was used as an opportunity for institutional 
renewal and as a tool for understanding and improving 
the college's image within and without the college. 

Topical Case Study 
Moderator: James Stager, Millersville Univ. of PA 



The college-fit theory and cognitive consistency and 
dissonance theories could offer some explanation on 
student attrition. This study will use three years of 
Admitted Student Questionnaire data to develop college 
image types and learn which pre-enrollment image type 
is most likely to predict the future enrollment behavior. 

Paper 

Moderator: Eleanor Swanson, College of New Rochelle 
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Putting the Real Focus 
Into Focus Groups 

Anita Dubey 

Mike Fusco 

Director 
Marketing, Bentley College 

2:55pm - 3:35pm 
Evelley 



A Review of the Writing 
Sample as an Index of 
Program Effectiveness 

Stanley S. Jacobs 

Office of Planning and 
Institutional Research 
Villanova University 

Mary Ann Ausetts 

Institute for Survey Research 
Temple University 

2:55pm - 3:35pm 
Abenia 



Influence of Background 
Variables on Students' 
Evaluations of Faculty 

Stuart L. Rich 

Director 
Institutional Research 
Georgetown University 

2:55pm - 3:35pm 
Triuna A 



Analysis of Funding Patterns 
for Federally Sponsored 

Research in Science 
& Engineering 1981-1991 

James Trainer 

Director 
HEDS 

Franklin and Marshall College 

2:55pm - 3:35pm 
Triuna B 



3:35pm - 3:45pm 



Using videotape clips, props, sample topic guides, and 
other collateral material, this presentation will demon- 
strate how to execute focus groups quickly, cheaply, and in 
a variety of settings. Examples from over 30 groups, 
conducted over an 18 month period, will be used to 
demonstrate how to satisfy objectives from publications 
development to customer service. This session will be of 
interest to those who would like to have a quick overview 
of focus groups and will also provide helpful hints to those 
who have previously conducted focus groups. 

Demonstration 



A review of research concerning the reliability, validity 
and useability of the writing sample as a special case of the 
essay examination was presented. The period since 1965 
was emphasized. Of particular concern was the utility of 
the writing sample as a measure of the effect of the under- 
graduate experience. 

Paper 

Moderator: James Ritchie, University of Pittsburgh 



The literature on students' evaluations of faculty yields 
conflicting evidence on the influence of background 
variables (course level, class size, discipline, grades sex 
and rank of faculty, e.g.) on ratings. Two years of course 
evaluations were examined to determine the relationship 
of these variables to faculty evaluations at a highly selec- 
tive private university. 

Paper 

Moderator: Marion Pagano, Columbia University 



This paper provides a longitudinal analysis of federally 
funded research in science and engineering for the period 
1 981 -1 991 . It examines the a lloca tion of funds to institutions 
over time and explores shifts in the sources of funds vis-a- 
vis various federal agencies. Data are sorted and analyzed 
by 13 agencies and 8 disciplines. The paper serves as an 
introduction to theuse and availability of federal databases. 

Paper 

Moderator: Walter Liss, Tufts University 



Break (Conference Center Lobby) 
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Monday 



NEAIR Conference Program 

The Sagamore, Lake George, 1993 



Time to Degree Completion 

Wendell G. Lorang 

Associate Director 
Institutional Research 
SUNY - Albany 

Craig Billie 

Associate 
Institutional Research 
SUNY Central Administration 

3:50pm - 4:30pm 
Evelley 



Developing Reliability 
Data for the ASQ Plus 



Ellen Armstrong Kanarek 

Program Director 
Applied Educational 
Research, Inc. 

3:50pm - 4:30pm 
Abenia A 



Trends In Campus Crime 



Bruce Szelest 
Associate 
Institutional Research 
University at Albany 

Fredericks Volkwein 

Director 
Institutional Research 
University at Albany 

3:50pm - 4:30pm 
Abenia B 



This paper presents a statistical profile and analysis of the 
differences between students who completed a bacca- 
laureate degree within four years and those students who 
took longer than four years to graduate. This study 
presents the methodology and results for nearly 12,000 
bachelor's degree recipients in the State University of 
New York system in 1991-92 and compares these results 
with those from the University at Albany, a large research 
university within the SUNY system. 

Paper 

Moderator: George T. Force, Slippery Rock Univ. of PA 



This presentation describes the steps involved in evaluat- 
ing the consistency of the ratings of the same colleges by 
the same students responding to Admitted Student 
Questionnaire Plus surveys mailed by two different 
colleges. Some of the issues examined include: identifying 
students completing more than one questionnaire; setting 
up the data file; choosing appropriate comparisons; 
interpreting the results. 

Topical Case Study 
Moderator: Richard C. Heck, Colgate University 



This research merged several national databases 
containing federal crime statistics, community demo- 
graphic data, and campus characteristics. The study 
displays the trends in campus crime since 1974, and using 
1990 data from over 390 campuses, examines the 
relationships between campus crime and college 
characteristics. 

Paper 

Moderator: Ann L. Henderson, Skidmore College 
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NEAIR Conference Program 



Monday The Sagamore, Lake George, 1993 



A Critical Multiplist Evaluation 

of Developing Reading 
Instruction at Suffolk Com. Col. 


Three quasi-experimental studies are presented which 
lend support for the positive effects of developmental 
reading courses on reading comprehension levels. Study 
1 used a regression-discontinuity design to test the effects 
of the developmental reading course on overall grade 
point average. Possible "mortality" bias was ruled out in 
Study 2 using a nonequivalent control group design. 
Study 3 used a single group pretest-posttest design to 
assess the effectiveness of the developmental reading 
course on improving reading comprehension skills. 
Significant pre- to post-test gains were found. 

Paper 

Moderator: Larry Metzger, Ithaca College 


Anthony Napoli 

Suffolk Community College 
and Department of Psychology 
SUNY at Stony Brook 

Paul Wortman 

Department of Psychology 
SUNY Stony Brook 

Christina Norman 

Department of Psychology 
SUNY Stony Brook 


3:50pm - 4:30pm 
Triuna A 




Whv Do Students Seek / 
Employment? / 
Summary Results of National 
Student Survey Responses 


1 IlC dlUvly L/l CdCllld i CaUUj VJl lllC lNClllUllCll u»lUV*Cl It 

Employment Survey, which involves over 4,500 responses. 
The study found that stud ents seek employment for finan- 
cial as well as educational/career enrichment purposes. 


Yuko MuJ^getta 

Spninr Rpseaix*h Associatp 
Office of Financial Aid and Student 
Employment 
Cornell University 


Paper 

Moderator: Thomas Flaherty, Central CT State Univ. 


/ 3:50pm - 4:30pm 
/ Triuna B 




4:30pm - 5:30pm 


Team Case Study Groups 

(Conference Center - Bellvue) 


6:00pm - 7:00pm 


Reception, Cash Bar & Hors D'Eouvres 
(Sagamore Dining Room) 


7:00pm - 12:00pm 


Dinner and Karaoke 

(Sagamore Dining Room) 
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Tuesday 



NEAIR Conference Program 

The Sagamore, Lake George, 1993 



Tuesday 



6:30am - 7:30am 
7:30am - 8:30am 



Robert Morse 

U.S. News & World Report 
7:30am - 8:30am 
Evelley 



8:00am - 9:00am 



Student RighMo-Know 
and Graduation Rate Data 

Roslyn Korb 

Chief of Cross-Sec. Studies Branch 
NCES 
Nancy B. Schantz 
Statistician 
Cross-Sectional Studies Branch 
NCES 

Susan G. Broyles 

Section Head of 
Student Surveys Inst. Studies 
Branch, NCES 

9:00am - 10:00am 
Evelley 



No Pain, No Gain: How One 
College Emerged Stronger From 
the Fiscal Crisis 

Craig A. Clagett 

Director 

Institutional Research & Analysis 
Prince George's Comm. College 

9:00am - 9:45am 
Abenia 



Organizational Sources of 
Disparities Between Male and 
Female Faculty Salaries 

Catherine E. Regan 

Research Associate 
SUNY System 
Fredericks Volkwein 

Director 
Institutional Research 
University at Albany 

9:00am - 9:45am 
Triuna A 



November 9 

Fun Run/Walk (Meet at Main Hotel Entrance) 
Breakfast (Conference Center - Nirvana) 

Bring your breakfast and come and discuss the most recent 
U.S. News' "America's Best College Values". All are welcome. 
Bring your criticisms, compliments, questions, or sugges- 
tions of any kind. 

Conference Evaluations (Invited InterviewsKNirvana) 

This presentation will address the current status of the SRK 
regulations and related compliance issues. In addition, 
presenters will discuss plans for pretesting the National 
Center for Education Statistics' (NCES) Graduation Rate 
Survey and incorporating the survey into the IPEDS data 
collection cycle. Audience participation is encouraged. 

Panel 

Moderator. Jane Zeff, William Paterson College 



Can a college benefit from a ten percent budget cut? After 
describing the measures adopted by a Maryland community 
college to meet this emergency, including introduction of 
an unusual differential fee structure based on discipline 
costs, the paper argues that the fiscal crisis provided the 
impetus for implementation of a number of reforms long 
contemplated but never adopted by the college. Implica- 
tions of this "crisis as catalyst" model for planning theory 
will be discussed. 

Paper 

Moderator: Stuart L. Rich, Georgetown University 

This research explores both organizational and individual 
sources of salary differences between male and female 
faculty. Drawing upon a database of 6,536 full-time faculty 
at a variety of research, doctoral, comprehensive, medical, 
and two-year institutions, the study uses multivariate 
analysis to examine the influences of organizational size, 
wealth, and complexity (in addition to the traditional 
measures that indicate a faculty member's education, work 
experience, rank, and discipline). 

Paper 

Moderator: Linda M. LeFauve 
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Tuesday 



NEAIR Conference Program 

The Sagamore, Lake George, 1993 



Outreach Programs and Their 
Varying Impact on Key Market 
Segments 

David J. Costello 

Dean 

Enrollment Planning 
Newbury College 

9:00am - 9:45am 
Triuna B 



How Insensitive Can You Get? 
Further Analysis of the 'Best 
Colleges' Rankings 

Michael McGuire 
Director 
Institutional Research 
Franklin & Marshall College 

9:55am - 10:35am 
Abenia 



Analytical Techniques for 
Studying Student Retention 

Anne Marie Delaney 

Director 
Program Research 
Boston College 

9:55am - 10:35am 
Triuna A 



Topical Case Study on Salary 
Compression and Inversion 

Denise A. Krallman 

Institutional Research Analyst 
Budgeting, Planning & Analysis 
Miami University 

9:55am - 10:35am 
Triuna B 



10:35am - 10:45am 
10:45am - 12:00noon 
1:00pm - 4:00pm 



Theoutreach efforts of many institutions are extensive and, 
more importantly, expensive. With each passing year, the 
recruitment effort among colleges for qualified students 
has increased. What has not maintained pace with this 
increased effort, is the management and assessment of 
specific outreach programs. This paper analyzes how 
various outreach programs impact key market segments 
(quality applicants, financial aid applicants, housing 



applicants, etc.). 



Paper 



Moderator: Yun Kim, Goucher College 

This session will feature the results of sensitivity analyses 
of the 1992 'Best Colleges' data for the National Liberal Arts 
category. Shifts in institutions' ranks were observed under 
three alternative weighting scenarios constructed from 
data obtained in a 1992 survey of college presidents; 
admissions directors, and academic deans. Explanations 
for the shifts in rankings will be offered, and the implica- 
tions of the 'Best Colleges' model's instability for both the 
institutions affected by those shifts and the consumer au- 
dience will be discussed. 

Paper 

Moderator: Joseph Revelt, Shippensburg Univ. of PA 

The purpose of this paper is to present the design and 
analytical techniques employed in retention studies that 
have produced information responsive to the questions 
and concerns of university administrators. Proposed 
enhancements to these studies - to include socially relevant 
variables and to meet anticipated 'Right-to-Know' infor- 
mation requirements - will also be discussed. Three topics 
to be addressed in the paper include: Data Requirements 
for a Longitudinal Retention File, Logic for Developing 
Computer Programs, and Analytical Techniques for Pre- 
dicting Retention. p 

Moderator: Ellen Armstrong Kanarek, Applied Ed. Rsch. 

A study was conducted this year at Miami University to 
look at the issue of salary compression (and in an extension 
of this study - salary inversion). Discussion will center on 
how best to identify salary compression and inversion so 
that inequalities can be addressed. 

Topical Case Study 
Moderator: Eleanor Fujita 



Break (Conference Center Lobby) 
Team Case Study Presentations (Conf. Ctr. - Bellvue) 
Steering Committee Meeting (Evelley) 
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1992-93 Steering Committee 
Officers Members- At-Large 



President 
Dawn Terkla 

Director of Institutional Research & Planning 
Tufts University 

President-Elect 
Michael McGuire 

Director of Institutional Research 
Franklin & Marshall College 

Secretary 
Mark Eckstein 

Director, Assessment and Institutional Research 
Genessee Community College 

Treasurer 
Wendell Lorang 

Assistant Director, Institutional Research 
SUNY - Albany 



Jennifer Brown 

Director of Institutional Research 
Connecticut State University 

Albert Lefebvre 

Dir. of Institutional Studies & Student Records 
Clark University 

Marian Pagano 

Director, Planning and Institutional Research 
Columbia University 

Karen Bauer 

Senior Research Analyst 
University of Delaware 

James C Fergerson 

Director of Institutional Research 
Bates College 

James L. Ritchie 

Assistant Director, Institutional Research 
University of Pittsburgh 



Past President Membership Secretary 
Larry Metzger Brenda Bretz 

Director of Institutional Research Assistant Registrar 

Ithaca College Dickinson College 

1993 Conference Chairs 

Conference Program 
Marjorie E. Wiseman 
Director of Institutional Research 
Northeastern University 

Local Arrangements 
Diane Cuneo 

Director of Institutional Research 
Smith College 

Publications 
Jane Price 

Assistant Director of Institutional Research 
Franklin & Marshall College 




Examples of How Institutional Research Can Help 
Campus Administrators with Their Survey Research Needs 

Karen W. Bauer, Ph.D. 
Senior Research Analyst 
Office of Institutional Research & Planning 
The University of Delaware 



At most institutions, institutional research staff members act as 
consultants to other campus administrators for departmental or other survey 
needs. This consultancy role in survey research is only one, but a very 
important role that many IR members embrace. Involvement in the design, 
analysis, and completion of a summary report can be a pleasant challenge. 

Over the past year, I have been asked to become involved in the design 
and analysis of several surveys that were eventually distributed to specific 
populations within the campus community. I offer, not as an expert, but as a 
dedicated institutional researcher, the "trials and tribulations" of my 
experiences with these research projects with the hope that you might learn 
something from my experiences. 

Like many Institutional Research offices today, our total number of 
staff members has been reduced. Expanding responsibilities in budget 
planning and support, enrollment management, and completion of the myriad 
internal and external ad-hoc surveys (including annual surveys from such 
places as Peterson's Guide, NASULGC, IPEDS, NCAA, and regional accrediting 
associations) all compete for each staff member's time and energies. In our 
Office of Institutional Research & Planning at the University of Delaware, the 
four professional staff members have created a team approach in order to 
accomplish our tasks. With a tenure of at least five years, we have each 
created our own niche or "specialty" area. Whenever possible, our director 
tries to assign tasks that fit within our specialty. There are advantages to such 
strategic assignments- efficiency and accuracy of task completion, and 
happiness of the staff member to utilize and strengthen his or her personal 
knowledge and skill. 

Recent Survey Projects 

During the 1992-93 academic year at UD, I coordinated the design and 
analysis of the following suivey projects: 

Fall 1992 Freshman Survey 

Parents Evaluation of New Student Orientation 

Preview Day 

CORE Alcohol and Drug Use Survey 
Classroom Participation Survey 
Salaried Staff Survey 
Part-Time Child Care Needs 

Personal, Career, and Learning Skills Needs Survey 
Campus Climate Scale 
Health Services Evaluation 
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Sexual Experiences (Harassment) Survey 

Through four examples, I will share the sequence of events that 
transpired from the time I received the initial phone call requesting help 
through completion and distribution of a summary report. Throughout my 
work on these projects, I have learned a great deal, and I offer you some of the 
positive and negative points learned from each. 



Parents' Evaluation of New Sturipnt Orientation 

M The University of Delaware, the IR Office works closely with the 
■\dmissions Office to share data, produce enrollment projections, and complete 
a number of external surveys (e.g., US News & World Report annual survey, 
Money Magazine, etc.) In a collegial spirit, our office has for many years 
helped the Admissions Office with evaluation needs such as those for Campus 
Tours, Preview Davs, and New Student Orientation. A copy of the 1992 Parents' 
Evaluation of New' Student Orientation is attached in Appendix 1. 



Sequence of Events 



1. Received call from NSO Director requesting help 

2. Gel last vear's NSO survey from files 

3. Meet with NSO director - determine if we can edit last year's survey or 

need to completely redesign 

4. Draft survev, get final approval from NSO director 

5. Share draft' of survey with director of data entry; get her approval 

6. Edit last vear's SPSS program 

7. Throughout summer NSO, receive daily/ weekly bundles of completed 

survevs 

8. Manually' scan to see that surveys have been completed correctly; 

collate open-ended comments 

9. Send to data entry 

10. Receive data files from data entry 

11. Combine data files and run SPSS to analyze data 

12. Share initial findings with IR staff 

13. Share initial findings with NSO director; determine if ant specific 

analyses desired. Usual analyses include frequencies and 
crosstabs. 

14. Draft summary report 

15. Share summarv report with IR staff 

16. Submit final summary report to NSO Director who will in turn share 

with her director and orientation colleagues. 

Less Than Positive Experi e nces With This P roject 

1. New data entry procedures required me to extend my efforts at 
accurately aggregating data files 
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Positi ve Expp ripnrps With This Projeci 

1. Survey is easy to complete, easy to scan for errors 

2. Designed for ease in data entry. 

3. Survey is relatively the same from year to year, so my start up time is 

reduced to a minimum. 

4. NSO staff tally open-ended comments, so again my time involved in 

analvsis is reduced. 
5 NSO staff is especially gracious and appreciative of the summary 

report that is written, but feels free to comments on the report. 



II. CORF Mmhnl and Drug Use Survey 

Seq uence of Events 

I. Dean of Students and Vice President for Student Affairs asked how we 

as an institution could look at this problem 
•> l checked our files, gathered info from library, ask IR colleagues, to 
learn that nothing had been done on our campus for at least past 
10 vears 

3. I recommended that we begin by gathering baseline aata 

4. Began literature review- obtained relevant documents from library, 

"Dean of Students, other colleagues . 

5. Called friends at several other campuses to see what they were doing , 

asked for copies of surveys they had used 
G. Visited George Mason Univ. Center for Drug & Alcohol Lse; learned of 
Washington, DC. area consortium on College Alcohol l se 

7. Made decision to use CORE Alcohol & Drug Use Instrument - See 

Appendix 2 

8. After discussions with VP Student Affairs and Dean of Students, 

prepared draft of proposal to go to President 

9. Proposal finalized and signed by VP Student Affairs. Dean of Students. 

Director of IR, and me. 

10. Received approval with no modifications from President 

II. Requested and received approval from University Human Subjects 

Committee 

12. Ordered survevs, envelope*^ mailing labels 

13 Selectively chose approxiniately 20 classrooms for survey 

administration. Specifically looked for classes with large 

enrollments and variety of majors. 
14. Verified students enrolled in these classes to ensure representation 

of all students by sex, class level, ethnicity ,and major. 
15 Sent personalized letter to each faculty member asking for 

admittance to administer survey during first or last 15 minutes ot 

class during specified week. 

16. Most said ves, u few said no- so had to choose a couple more classes. 

17. Reverified' breakdowns by class, sex, ethnic, and major 

18. Created schedule for entry into each classroom 

19. Replied to each faculty member with date and time 
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20. Administered survey; elicited help from office work study student on 

a few occasions when two classes were scheduled at once 

21. Manually scanned sheets to check for accuracy of completion 

22. Mailed surveys to company in Minnesota for scoring 

23. About two weeks later, received diskette of data and printed copy of 

frequencies 

24. Completed additional analyses- mostly frequencies, crosstabs, T-test, 

MANOVA 

25. Shared initial findings with IR staff, then VP Student Affairs 

26. Submitted summary report to VP Student .Affairs 

27. Shared summary report with faculty who requested it 

28. VP created task force to further examine campus practices, 

programs. 

Less Than Positive Experiences With This Project 

1. Very time consuming during scheduling and week of administration 

It took a great deal of time and effort to choose classes, verify 
breakdowns, send letters to faculty, coordinate logistics of 
entering each classroom. I was literally running from one 
classroom to another! 

2. Didn't anticipate everything-- Learned on first day that not everyone 

had pencils, so had to run out and buy pencils 

3. Had to deal with a few faculty with uncooperative attitudes 

4. Frustrations with little action occurring after survey results 

distributed. 

Positive Experiences With This Project 

1. Getting President's approval was a key in this project's success 

2. I worked hard to get permission to administer in classes; felt a sense 

of satisfaction; allowed me to more closely control atmosphere 
under which survey was completed. 

3. Was good to work with faculty- collegiality, politically 

4. We helped Student Affairs obtain good baseline data 

5. Having normative data was very helpful- showed where UD students 

were above and below other norms 

6. Was a relatively inexpensive survey 

7. Developers at So. Illinois were especially helpful on phone 

8. Purposefully built in three year timeline for follow-ups 

9. I learned a lot about college student drug and alcohol use 

10. I was asked to present study results to a graduate class 
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TIT. Gender Differences In Classroom Participation 
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Sequence of Events 

1. Attended meeting of Commission on Status of Women in which 

student constituency discussed several projects of interest 

2. I helped them narrow and prioritize issues; plan to address chilly 

climate issues . 

3. Did extensive literature review to understand chilly climate; obtained 

relevant articles and studies 

4. Obtained similar study that was conducted at UD in 1983 

5. Developed and refined methodology- See Appendix 3 

6. After two revisions obtained approval from Psychology Department 

Human Subjects Committee 

7. Sent letter to all department chairs informing them that this 

observation would take place 

8. Completed pilot study - Spring 1992 

9 Shared pilot study results with Commission on Status ot \\ omen 

10. Requested and obtained approval from Human Subjects Committee 

for full study- Fall 1992 

11. Obtained additional counsel from IR director and senior level 

administrators 

12 Sought found, and trained three student research assistants 

13. Used SPSS to randomly select classes with specific criteria; randomly 

assigned research assistants to classes 

14. Monitored observers throughout the week of observations 

15. RAs returned completed sheets to me 

16. Some sheets weren't complete, so had to follow-up with RAs 

17. Completed inter-rater reliability checks 

18. Completed full analysis 

19. Shared findings with IR staff . 

20. Submitted summary report to Commission on Status of Women 

Less Than Positive Experiences with This Project 

1 Very time consuming- some weeks about 20-25% of my time 
?. During pilot study, one department chair telephoned to inquire about 
the study and threatened to personally submit my name to faculty- 
senate for unethical behavior 

3. Frustrating when student observers didn't live up to my expectations 

4. Frustrations with primary student researcher who presented 

findings from the study at a campus research conference 

Positive Experiences With Th is Project 

1. Was very exciting to be doing 'experimental' research 

2. I felt that this study was very important and could contribute to the 

status of women in higher education 
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3. I was completing past of my assigned duties in providing the 

Commission on Status of Women with data and related 
information. 

4. By modeling previous research, it was easier to make refinements to 

already existing methodology. Also- because this was so 
politically hot, the previous study in 1983 served as a precedent. 

5. I became much more aware of the political climate on campus; 

changed some of my naivete 

6. This research helped me become really interested in the chilly 

climate issue and has moved me to a new level of interest in 
gender differences. 



IV. Longitudinal Study 
Sequence of Events 

1. Completed a follow-up with CSEQJn Spring 1992, really saw the 

benefits of longitudinal studies. 

2. Continued to read, did additional literature reviews, saw the benefits 

of longitudinal work 

3. Submitted grant proposal to UD for grant $$- was turned down- Fall 

1991 

4. Sent letter to Provost's Office requesting funds for study- was turned 

down- Spring 1992 

5. 1R Director continued to support the idea, encouraged me to keep 

trying 

6. Submitted grant proposal to NEAIR grants committee- Fall 1992 

7. Received word that I would receive funding from NEAIR - Spring 

1993 

8. Reviewing costs over 4-5 years, I decided to change primary 

instrument; asked NEAIR grants committee for approval to 
change from more expensive published instrument to one 
developed at SUNY-Albany-- See Appendix 4 

9. Committee Ok'd changes, made award. 

10. Once received $$, took check to University grants office, they set up 

account for my project. 

1 1 . Received OK from data entry 

12. Ordered copies of survey to be reproduced 

13. Surveys put in freshman packets, completed at New Student 

Orientation 

14. Completed surveys sent to me 

15. Surveys manually scanned for accuracy; four letter major code 

entered 

16. Surveys sent to data entry 

17. Created SPSS command program for analysis 

18. Received keyed surveys from duu entry 

19. Have just begun analysis 
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Less Than Positive Experiences with This Project 



1. Requires incredible tenacity, persistence, dedication to the project 

2. Getting turned down for grant funds was a downer 

3. Thought it would be easier to get funding- costs of project required 

me to change from published instrument (with national norms) 
to one that was locally-developed 

Positive Experiences with This Project 

1. Am pleased to begin a study that I've wanted to do for some time- this 

gives me new energy 

2. I was able to obtain large initial sample through NSO administration 

3. We will know a great deal about this cohort of students - not only 

demographicallv, but hopefully we will also gain insights into 
die process of learning, how students change from year to year 

4. I feel lucky to be one of relatively few researchers who get to do such 

a longitudinal study 

5. Creates possible new ties with inter-institutional data comparisons. 
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Appendix 1 
1990 HEW STUDENT ORIENTATION 

PARENTS' EVALUATION 



Thank you for taking the time to complete this evaluation. Your responses will help us in planning future simmer 
programs. Read each guestion carefully and write the number of your response in the box at left. Please leave 
this evaluation form in one of the boxes at the back of the auditorium before leaving today. 



TODAY'S DATE (for numbers below 10 enter 01, 02, etc.) 



MONTH DAY 



A. Person filling in this questionnaire is: 

1- Hother 3, Both parents 

2. Father 4, Other (Please specify) 

B. Are you a resident of: 

1. Delaware 

2. Out-of-state 

C. Ho* satisfied were you with the following aspects of the New Student Orientation? 

Using the scale belov, enter the appropriate number (1-5) corresponding to your rating 
in the box at the left hand side of the event, 

1. Very Satisfied 3, Somewhat Satisfied 5. Did Not Attend 

2, Satisfied 4, Very Dissatisfied 

Welcome and Announcements 

Academic Life: Creating an Intellectual Adventure 

Student Affairs: Maximizing the University Experience 

Counseling and student Development: Easing Students' Transition 

Residence Life: Building a Dynamic Community 

Off -campus Living: Becoming a Part of the Student Body 

Lunch 

Students: "Tell It Like It Is n 

Dollars and Sense: Answering Questions About Paying the Bills 
University Honors Program (Parents of students admitted to the Program) 



(Cols 1-16) 




(Continued on reverse side) 
22 3 



Form 191 



Core Alcohol and Drug Survey 

For use by two- and four-year institutions 



FIPSE Core Analysis Grantee Group 



Please use a number 2 pencil. 



Processed by. UCS/Office of Measurement Services 
University of Minnesota 
2520 Broadway Drive - Room 130 
St. Paul. MM 55113 




: C ®®®&® ®X3) <S>®®% 
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1. Classification: 

Freshman O 

Sophomore O 

Junior O 

Senior O 

Grad/professional O 

Not seeking a 

degree O 

Other O 



5. Gender: 

Male O 

Female O 



2. Age: 



® ® 

© 0 

® ® 

© ® 

® ® 

® ® 

® ® 

® ® 

® ® 

® ® 



Ethnic origin: 
American Indian/ 

Alaskan Native O 

Hispanic Q 

Asian/Pacific Islander. . O 
White (non-Hispanic) . . . O 
Black (non- Hispanic) . . . O 
Other O 



6. Is your current residence 
as a student: 

On-campus O 

Off-campus O 



9. Approximate cumulative grade average: (choose one) 

ooooooooooooo 

A+ A A- B+ B B- C+ C C- D+ D D- F 



10. Some students have indicated that alcohol or drug use at parties they attend in and 
around campus reduces their enjoyment, often leads to negative situations, and 
therefore, they would rather not have alcohol and drugs available and used. Other 
students have indicated that alcohol and drug use at parties increases their 
enjoyment often leads to positive situations, and therefore, they would rather have 
alcohol and drugs available and used. Which of these is closest to your own view? 

Have available Not have available 

With regard to drugs? O O 

With regard to alcohol? O O 



Marital status: 

Single 

Married 

Separated 

Divorced 

Widowed 



• O 

• O 

• O 

• O 
■ O 



Are you working? 

Yes, full-time 

Yes, part-time 

No 



• O 

• O 
■ O 



8. Living arrangements 

A. Where: (mark best answer) 
House/apartment/etc. . . O 

Residence hall O 

Approved housing O 

Fraternity or sorority . . - O 
Other O 

B. With whom: 
(mark ail that apply) 

With roommate(s) O 

Alone O 

With parent(s) O 

With spouse O 

With children O 

Other Q 



11. Student status: 

Full-time (12+ credits). . . O 
Part-time (1-11 credits).. O 



13. Place of permanent 
residence: 

In-state O 

USA, but out of state. . . O 
Country other than USA. O 



no 

.o. 
.o. 



12. Campus situation on alcohol and drugs: yes 

a. Does your campus have drug and alcohol policies? . . . O . 

b. If so, are they enforced? O . 

c. Does your campus have a drug and alcohol 

prevention program? O O 

d. Do you believe your campus is concerned about 
the prevention of drug and alcohol use? Q. 

e. Are you actively involved in efforts to prevent drug 
and alcohol use problems on your campus? O. 



don't know 



.o. 
.o 



o 
o 



14. Think back over the 
last two weeks. How 
many times have you 
had five or more 
drinks* at a sitting? 

None O 

Once O 

Twice O 

3 to 5 times . . . o 

6 to 9 times O 

10 or more times O 



15. Average # of 
drinks* you 
consume a week 



(If less than 



© 



*A drink is^ a bottle of beer, a gjass 
of wine/a* wfne cbbfer, £ shot *~ 



• ' glass of liquor, or a mixed drink^j..-- 



10, code ® 
answer as © 
01, 02, etc.) © 

© 
© 

© 
© 



© 
© 
© 
© 
© 
© 
© 
© 
© 
© 
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16. At what age did you 
first use. . . (mark one 
for each line) 

a. Tobacco (smoke, chew, snuff) 

b. Alcohol (beer, wine, liquor)* 



ooooooooo 
ooooooooo 



c. Marijuana (pot, hash, hash oiP QOOOOOOOQ 



d. Cocaine (crack, rock, freebase) OOOOOOOOO 

e. Amphetamines (diet pills, speedJOOOOOOOOO 

f. Sedatives (downers, ludes) OOOOOOOOO 



g. Hallucinogens (LSD, PCP) 

i. ODiates (heroin, smack, horse) 

i. Inhalants (glue, solvents, gas) 



ooooooooo 
ooooooooo 
ooooooooo 



j. Designer drugs (ecstasy, MDMA) OOOOOOOOO 
k. Steroids OOOOOOOOO 
I. Other illegal drugs OO.QOOOQOO 
•Other than a few sips 



BEST COPY AVAILABLE 



Appendix 3 
CLASSROOM INTERACTION STUDY 
TALLY SHEET 



Today's Date / / 



Upper Lower 



Course Classification: Level: 

Length of class: 50 minutes 75 minutes 150 minutes 

Sex of Instructor: Male Female 

Number of students attending: Male Female 

Student contributions offered - hands raised to speak. 

Men 

Women 
Student contributions - made 

Men Long (S seconds or longer) 

Short (less than 5 seconds in length) 
w oraen Long (5 seconds or longer) 

Short (less than 5 seconds) 

Feedback From Instructor 

To Male Student s: Positive 

Negative 
Neutral 
None 

To Female Students : Positive 

Negative 
Neutral 
None 

Other Instructor or Student Remark* On the back, please describe briefly any positive, negative, or 
neutral comments and other behaviors made by instructor or student. 

(Revised Fall 1992) 

o 
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Appendix 4 
FaU 1993 INCOMING STUDENT SURVEY 



You have been selected as part of a special group of first-year students to participate in a longitudinal study at the 
University of Delaware. The information below is being collected by The Office of Institutional Research in order 
to gain a better understanding of college students and to improve the quality of programs and services. Your 
voluntary participation in this study will help us achieve these goals. Please complete the information below 
completely and accurately. All responses are confidential and data will be reported only in the aggregate. Each year 
you will be recontacted and asked to complete a survey about your experiences at Delaware. We ask that you 
include your student ID (SSN) so that we can keep all of your information together. Thank you for helping us learn 
about your college experiences at Delaware. 



1. Student ID# 



Are you a: 



New Freshman 



Transfer Student 



2. Will you be living on campus this fall? (Circle one) A. Yes B. No 



3. 



I 

r ® 

Ierlc 



7. 
8. 



How well do you feel that your high school prepared you in the following areas? (Please circle one number in 
each row.) 

Not at Extremely 

All Moderately Well 

a. Mathematical skills 1 2 3 4 5 

b. Writing and composition 1 2 3 4 5 

c. Foreign languages 1 2 3 4 5 

d. Sciences 1 2 3 4 5 

e. American and World History 1 2 3 4 5 

f. Social issues (race rel., drug abuse, etc) 1 2 3 4 5 

g. Art/Music/Drama 1 2 3 4 5 

h. Study skills 1 2 3 4 5 

During your last year of high school, approximately how many hours per week, on the average, did you: 

spend on homework: read school-related books: 

spend at the library: use a computer to play games: 

use a computer for school or homework: 

What was your high school GPA? 



Students come to college with different goals. Please indicate 
how important each goal is for you by circling the number. 

a. To gain a broad, liberal arts education and 

appreciation of ideas 

b. To gain knowledge and skills directly applicable 

to a career 

c. To learn to think creatively and analytically 

(L To learn more about myself, my values, 

and my life's goals 

e. To learn more about other languages and cultures . . . 

What is your major? 



None 



Level of Importance 



Moderate 



Essential 



2 
2 

2 
2 



3 
3 

3 
3 



4 
4 

4 
4 



5 
5 

5 
5 



What is the highest degree you expect to obtain? (Circle One) 

1. Bachelor's degree 

2. Master's degree or equivalent (MA, MS, MBA, MP A, etc.) 

3. Terminal degree or equivalent (PhD, EdD, MFA, MD, JD, PsyD) 

Do you feel confident that you have adequate finances to 

pay your University of Delaware bill throughout 

your baccalaureate program? 



Yes 



No 



25 
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Appendix 5 
Spring 1994 STUDENT SURVEY 



i 
■ 

Last fall you completed a questionnaire when you entered the University. We are asking you to give us additional I 
mation now that you are near the end of your second semester of study at Delaware. Your voluntary participation * 

I 
l 



information 

will help us improve the quality of programs and services, especially for freshmen and other new students. We need 
your name and social security number in order to follow-up with non-respondents and to link your responses to the ones] 
you gave us last fall. However, all responses will be held in strict confidence and used only in group summaries for 
research, planning and program purposes. Please return by April 15 in the enclosed envelope to the Office of 
Institutional Research. Thanks very much for your help. 

PLEASE ENTER YOUR RESPONSES ON THIS SHEET 



1. 



Student ID # 



During the past academic year, approximately how may hours 
per week, on the average, did you spend in these activities: 

Studying outside of classes and labs 

On-Campus Employment (including work/study) 

Off-Campus Employment 

Sports, intramurals & Physical fitness 

Clubs, organizations, student activities, volunteer services, 
fraternities, sororities (Excluding sports) 

Please estimate the NUMBER OF TIMES during this past year that you 
have met with a faculty member outside the classroom for each of 
the reasons listed. Count only conversation of 10 minutes or more. 

a. To discuss your academic program 

b. To discuss matters related to your future career 

c. To help resolve a disturbing personal problem 

d. To discuss intellectual or course-related matters 

e. To discuss a campus issue or problem 

f. To socialize informally 



In which academic field do you expect to major? 
(If undecided, please so state): 



Hours Per Wee k 
FALL ^SPRING 



Number of Times Met With 



A Faculty 
.Member 



Another Univ. 
Staff Member 



How many times during 1993-94 have you changed 
your plans for an academic major? NONE 

1 



6. How great was the change in your plans? 



7. How was your satisfaction with the University 
changed since you first entered Delaware? 



Complete 
Change 
1 



More 

Satisfied 
1 



ONCE 



Large 
Change 
2 



About 
the Same 
2 



TWICE 
3 

Moderate 

.Change 



Less 
Satisfied 
3 



THREE 
TIMES 
4 

Slight 
Change 
4 



FOUR OR 
MORE TIM ES 
5 

No 

Change 
5 
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No Fain, No Gain: 
How One College Emerged Stronger from the Fiscal Crisis 



Craig A. Clagett 
Director, Institutional Research and Analysis 
Prince George's Community College 



Unless challenged, every organization tends to become 
slack, easygoing, diffuse. It tends to allocate 
resources by inertia and tradition rather than 
results. Above all, every organization tends to avoid 
unpleasantness . And nothing is less pleasant and less 
popular than to concentrate resources on results, 
because it always means saying "Wo." 

Peter Drucker 



Introduction 

Historically, Prince George's Community College (PGCC) has 
been among the most cost-efficient community colleges in 
Maryland. Its overall cost per student has consistently been 
among the lowest in the state, a situation forced upon the 
College due to comparatively low levels of county support. 
According to past state guidelines, community colleges in 
Maryland were to receive 50 percent of their revenues from the 
state, 28 percent from their local jurisdiction, and 22 percent 
from students. The actual proportions have deviated from these 
guidelines. Statewide, the 16 locally-governed community 
colleges received 27 percent of their FY92 funding from the 
state, 37 percent from their respective counties, 3 5 percent 
from students, and 2 percent from other sources. Prince 
George's County contributed 2 6 percent of PGCC's budget in FY92. 
An analysis of county support for community colleges in Maryland 
found that, by most measures, PGCC received about half the level 
of local aid that peer colleges in neighboring jurisdictions 
received (Clagett, 1992). 

Largely because of the above situation, PGCC was hit 
especially hard by the state fiscal crisis of the early 1990s. 
State revenue shortfalls, combined with mandated Medicaid and 
welfare expenditures, led to substantial cuts in state aid to 
higher education institutions in Maryland. The impact on PGCC 
was felt through three consecutive years of mid-year state 
funding reversions: 



r O 
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Direct State Aid to PGCC 


Year 


Budgeted 


Reversion 


Received 


FY90 


$11,367,820 


$o 


$11,367,820 


FY91 


11,679,772 


822,912 


10,856,860 


FY92 


11,583 , 160 


2,768,655 


8,814,505 


FY 03 


11,588,980 


1,038,258 


10,550,722 


In addition, ii 
employee retin 
nearly $1 mill 


n FY92 the state 
ament systems fo 
ion. The colleg 


contribution tc 
r PGCC employees 
e had to fund ti 


) FICA and 

> was cut 

lis shortfall. 



As can be seen from the above table, the major budget hit took 
place in fiscal year 1992 . The state funding reversion for that 
year consisted of a nearly one-fourth reduction in direct aid, 
plus a similar reduction in the state's contribution to college 
FICA and retirement systems. This resulted in an effective 33 
percent cut in state assistance to PGCC, as the College had to 
make up the cut in the FICA/retirement contribution. In 
dollars, the total cut amounted to nearly $3.8 million, or ten 
percent of the College's total budget of $38.4 million. In 
addition to the state aid cut, the County eliminated a planned 
$1.1 million dollar increase in aid to PGCC, providing the 
College with the same local aid in FY92 that it had received in 
FY91. 

Meeting the Revenue Shortfalls 

To meet the immediate FY92 crisis , the College implemented 
several cost containment measures. These included a freeze on 
hiring, elimination of all conference travel, elimination of all 
professional development and training, and a freeze on all 
equipment purchases. Only essential purchases of supplies and 
materials in direct support of instruction or safeguarding life 
and property were permitted. 

An employee furlough plan was implemented. Twelve-month 
employees were furloughed seven days, and ten-month employees 
were furloughed five days. Employees lost an average of $1,000 
each in wages, with the pay cut spread evenly across all 
remaining paychecks. In addition, the rate of compensation for 
full-time faculty teaching summer session courses was reduced. 
To further reduce the funding gap, $150,000 in Capital Expense 
Reserve Fee revenues were transferred to the operating budget. 

The above expenditure reductions were insufficient for 
meeting the state funding cut, so the College was forced to 
raise revenues. This was done through the introduction of a new 
consolidated student fee. The origins and development of the 
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Instructional Services Fee (ISF) will be discussed in a later 
section. To sum up, the FY92 state aid reversion was 
accomplished as follows: 



i 
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Measures to Meet FY92 State Aid Reversion 


Cost containment 


$1, 300,000 


Employee furlough 


600,000 


Summer teaching pay rate reduction 


250,000 


Capital Expense Reserve Fund transfer 


150,000 


Instructional Services Fee (ISF) 


1,500,00 


Total 


$3,800,000 



The College's asking budget for fiscal year 1993 was the 
same as the year before, $38.4 million. Initial revenue 
projections indicated a likely deficit of $1.8 million. To meet 
this anticipated budget deficit, three strategies were 
implemented at the beginning of the year. A fourth strategy, 
elimination of programs and services, served as a contingency 
plan had the financial situation deteriorated further. 

The first strategy was administrative reorganization. 
Through reallocation of responsibilities, seven administrative 
positions were abolished. This reduction, from 41 to 34, 
represented a 17 percent decrease in administrative positions. 
The positions eliminated were: five deans (of computer and 
engineering technology, educational development, humanities, 
science and mathematics , and physical education) , and two 
directors (of admissions and career planning) . The 
reorganization essentially consisted of seven cases of two 
divisions or offices being collapsed into one (for example, 
English and humanities, previously with their own dean, were 
combined into one division) . 

The reduction in instructional deans was in accord with 
recommendations that surfaced seven years earlier during the 
College's self study prepared for Middle States accreditation 
review. The College's study group on organization, 

administration, and governance, six of its ten members faculty, 
recommended in its 1985 report that the role and number of 
instructional deans be examined, asserting that "reassessment of 
the role of the deans may result in fewer dean positions. 11 The 
Middle States visiting team, citing a perceived lack of academic 
leadership plus faculty "discontent with f ragmentation, 
territoriality, and lack of cohesion, 11 gave "unqualified 
support" for reducing the number of deans. However, follow-up 
discussions on campus , including a forceful written response 
from one of the deans (subsequently endorsed by several other 
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deans) , led to modest changes in responsibilities but no 
reduction in positions. 

The second strategy implemented to meet the projected FY93 
deficit was a voluntary resignation incentive program, commonly 
referred to as the "early out," offered to College employees 
with a minimum of 20 years of service. Of the 170 employees 
eligible, 17 or ten percent applied for the program and were 
approved. Ten faculty, three administrators, and four 
classified staff received incentive payments equal to half their 
annual salary in exchange for their early resignation. 

The third strategy for FY93 budget planning involved cost 
savings through "downsizing" efforts implemented by the vice 
presidents in each area. Though the measures adopted varied by 
area, common approaches included eliminating vacant positions 
from the operating budget and reducing hours of operation. 
Seventeen full-time and ten part-time classified staff positions 
were abolished. 

In addition to the above measures, the College entered FY93 
continuing the austerity program of no professional development, 
no conference travel, a freeze on hiring except positions deemed 
essential by the President, and no salary improvements. These 
cost containment measures, combined with implementation of the 
three strategies described above, produced the following savings 
to meet the projected $1.8 million deficit: 



Measures to Meet Projected FY93 Deficit 


Administrative reorganization 


$300,000 


Voluntary resignation incentive program 


400,000 


Area downsizing 


800,000 


Cost containment 


300,000 


Total 


$1,800,000 



Instructional Services Fee 

The primary justification for implementing the new 
Instructional Services Fee, or ISF, was to generate the revenue 
needed to meet the FY92 budget shortfall created by the state 
funding reversion. A iee was chosen, as opposed to an increase 
in tuition, because PGCC's high tuition had become a sensitive 
political issue. Due to historically low levels of county 
support, compared to peer community colleges in Maryland, PGCC's 
tuition was among the highest in the state — a fact that had 
become a focal point of some political attacks on the College 
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(Clagett, 1992). Instituting a reform in the College's fee 
structure enabled the institution to maintain its tuition 
unchanged at $55 per credit hour. The ISF was also introduced 
as a simplification of PGCCs increasingly complex set of fees. 
In addition to registration and student activity fees, the 
College had — prior to the ISF — 271 separate course laboratory 
fees, a capital expense reserve fee, a natatorium fee, plus 
three pending fees: an instructional support services fee, an 
instructional technology fee, and a parking fee. The ISF 
replaced all but the registration and student activity fees. 
But there was another motive behind the ISF. 

In September 1984, at the request of the vice presidents of 
instruction and finance, the institutional research office 
investigated several multi-tiered, cost-based tuition schemes. 
The proposed differential pricing concept based student charges 
on the actual cost of providing instruction in different 
disciplines. The argument was that students pursuing more 
expensive programs of study should pay more than those taking 
less expensive courses. The additional revenue generated by 
students in the higher-priced disciplines would be used to fund 
the costs associated with them, such as computing time, 
supplies, and equipment. In this way, programs could be kept up 
to date by the tuition revenues of the students directly 
benefiting. Cost-based differential pricing was seen by its 
proponents as both fair and as not too far removed from the 
existing pricing scheme which included a variety of course-based 
laboratory fees. However, the differential tuition idea didn't 
go anywhere. A major concern was that higher tuition might 
encourage students to shift to less expensive programs, driving 
up unit costs in the more expensive programs even further, 
perhaps to the point that socially-needed programs (such as 
nursing and allied health) might be terminated due to 
perceptions of excessively high costs. Not wanting to penalize 
students for their curriculum 'choices, plus institutional 
reluctance to be an innovator in this area ("no one else is 
doing it"), stopped the idea in its tracks. It took the fiscal 
crisis of October 1991 to revive the idea in different clothes. 

A discipline cost analysis system developed by the Maryland 
State Board for Community Colleges in 197 3 provided the 
foundation for the new fee. The SBCC programs yield total cost 
per FTE student for teaching classes in each instructional 
discipline. Total costs include faculty compensation, 

additional direct instructional costs such as laboratory 
supplies, and indirect or overhead costs allocated across 
disciplines in proportion to their respective shares of total 
college enrollment. Costs are assigned to individual course 
sections and then summed to the discipline level. Discipline 
unit costs are influenced by instructional methodology factors 
(class size, course contact/credit hour ratios, and supply and 
equipment costs associated with lecture/ laboratory/ clinical 
formats) and institutional history factors, such as discipline 
differences in the proportional use of adjunct faculty and full- 
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time faculty rank and salary profiles. Representative of 
discipline costs per FTE in FY93 at PGCC were $3,107 in remedial 
mathematics, $3,293 in business management, $3,625 in history, 
$4,508 in art, $4,893 in chemistry, $5,231 in electronics 
technology, $5,659 in nursing, and $7,118 in nuclear medicine. 



The Instructional Services Fee implemented in Spring 1992 
had three levels — $15, $20, and $25 per credit hour: less 
expensive lecture courses were assigned the $15 fee, limited- 
enrollment health technology courses were assigned the $25 fee, 
with laboratory courses typifying the mid-range at $20 per 
credit hour. The new fee structure cost a part-time student 
carrying seven credit hours an additional $91 to $136 per term, 
depending on the particular courses taken. 



Instructional Services Fee 
(per Credit Hour) 


Level I 


Level II 


Level III 


English 


Computer/Info Sys. 


Nuclear Medicine 


Management 


Engineering Tech* 


Nursing 


Math/ Engineering 


Languages/ EFL 


Radiography 


Criminal Justice 


Office Technologies 


Respiratory Ther. 


Philosophy 


Physical Education 




Social sciences 


Natural sciences 




Speech 






$15 


$20 


$25 



Approximately 70 percent of the annual credit hours in FY92 were 
in Level I disciplines, 27 percent in Level II, and three 
percent in Level III. Despite the increased costs to students 
associated with the ISF, enrollment in health technology has 
increased steadily since implementation of the new fee 
structure. 



Recruiting the Non-college-bound 

The administrative reorganization and new Instructional 
Services Fee addressed the immediate financial emergency facing 
PGCC in 1992-93. A third major proposal was intended to address 
the longer term financial outlook. Realizing that constrained 
public support would continue to exert pressures for higher 
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tuition and fees, the College was concerned about the ability of 
students to pay and what impact the higher charges might have on 
enrollment. The ISF had raised student charges to the point 
that for full-time students taking 15 or more credits PGCC was 
no longer less expensive than several campuses of the University 
of Maryland. Unlike the University, PGCC did not have a full- 
time tuition package but instead charged the per-credit hour 
rate regardless of the number of hours carried. Thus, while 
PGCC's per-hour rate was still less than the senior 
institutions, hours taken above 12 were "free 11 at the University 
while they added to the community college student bill at the 
set per-hour rate. The net result was that PGCC had lost its 
competitive price edge for full-time students. The old 
arguments aimed at full-time baccalaureate seekers, such as 
"come to the community college for your first two years and save 
money for your junior and senior year, " simply were untrue. 
While full-time students accounted for only a fourth of PGCC's 
credit headcount, they generated 44 percent of the College's 
credit hours. Significant losses in full-time students would 
clearly erode the student revenue base . Finance office 
projections suggested that student tuition and fees would have 
to acv. . t for half of PGCC's budget by fiscal year 2000* At 
current enrollment levels, tuition/ fee rates would have to more 
than double to meet this target. The College's administration 
was concerned about the impact of such increases on both 
enrollment levels and the institution's mission, particularly 
its commitment to educational access. How could the College 
meet its budget forecasts and constrain the rise in student 
tuition and fees? One answer that surfaced was to increase 
enrollment. Given its current average class size of less than 
20 students, the College could absorb additional enrollment 
without incurring excessive new costs. 

The College's director of recruitment had argued, for over 
a decade, that PGCC should more aggressively recruit County high 
school students who were not considering college. He was 
convinced, through years of working closely with the schools, 
that a substantial number of students could benefit from what 
the College had to offer. Such a recruiting strategy, if 
successful, promised to meet a social need, generate needed 
enrollment revenue, fulfill the College's access mission, and 
perhaps promote County economic development by expanding its 
productive workforce. In addition, data compiled by 

institutional research had documented that Prince George' s 
County had a low college attendance rate compared to 
neighboring, peer jurisdictions. To advocates of this strategy, 
the non-college-bound constituted a natural market for the 
community college to serve. However, for over ten years, the 
arguments against this idea had prevailed. Faculty, in 
particular, were concerned about increasing the number of 
unprepared and possibly disruptive students. A shift in 
resources to developmental education would be needed, difficult 
in a time of fiscal constraint. Increased demand on 
instructional support services would also require additional 
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funds ♦ Expanding the population of students needing 

remediation, already a majority, was seen as harmful to the 
College's external image* An influx of unprepared students 
would probably have a negative impact on student outcomes 
indicators, potentially a public relations problem as 
accountability measures gained greater visibility. Finally, the 
non-college-bound segment might be expensive to market, and such 
recruiting efforts probably had a low probability of success. 
After all, by definition this group was not interested in 
college and thus recruiting messages would have a hard time 
connecting. 

The budget difficulties of 1992-93 persuaded many members 
of the PGCC administration that a strategy to increase 
enrollment was necessary. The dean of enrollment management, 
joined by the director of institutional research, made the case 
in February 1993 that the only market with substantial growth 
potential was the currently non-college-bound. PGCC already 
"owned" the part-time market, enrolling the overwhelming 
majority of County residents attending as part-time 
undergraduates. The full-time, baccalaureate-seeking market was 
not promising due to the loss of PGCC's historical price edge, 
plus aggressive recruitment from senior institutions across the 
country ♦ (Prince George's County, containing the nation's 
largest black middle class, was a prime recruiting ground for 
colleges seeking African-American students.) The case was made 
that an outreach strategy, aimed at neighborhoods and high 
schools with low college attendance rates, and incorporating 
financial aid workshops, should be considered. In addition, the 
research office had developed a "lifestyle cluster" geo- 
demographic market analysis system (PG-TRAK®) that could help 
target recruiting efforts (Boughan, 1990) . A proposal to use 
the newly-created PG-TRAK® for targeting the non-college-bound 
in February 1990 had been rejected ♦ Three years later, in 
different financial circumstances, a similar proposal was 
adopted. As of November 1993, plans for such a strategy are 
being developed. 



Fiscal Crisis as Catalyst for Reform 

The fiscal crisis provided both opportunity and 
justification for implementing ideas that had been contemplated 
for a number of years . The reduction in instructional 
administrators, introduction of a differential student fee 
structure based on discipline cost analysis, and aggressive 
marketing of the non-college-bound — all had been proposed and 
debated but not adopted during the good times. Concepts of 
merit such as cost-based differential pricing, and tools of 
value like geo-demographic market research, had lain dormant due 
either to a lack of critical mass or active resistance by key 
members of the college community. The severe fiscal crisis 
precipitated by state funding cuts of unprecedented magnitude 
provided the catalyst for reforms that, on balance, have been 
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beneficial to the institution. The process of adjustment was 
difficult and painful. Employees suffered wage reductions and 
students had to pay more. But the mission of the college was 
preserved, even enhanced, and the quality of the teaching- 
learning experience maintained. Indeed, by emerging a leaner 
and fiscally stronger organization, the College was able to fund 
several new initiatives in FY94 while neighboring institutions 
were still struggling with cost containment programs. 



Shadow Plans 

The administrative reorganization, new fee structure, and 
targeted recruiting strategy can be characterized as components 
of a "shadow" plan that remained hidden from view but was 
available for implementation at the opportune time. Such a plan 
differs from the more visible public relations, bureaucratic, 
and action plans familiar to most planners in higher education. 

The public relations plan provides legitimation and the 
appearance of rational management to external audiences . 
Attractive college publications describe, integrated planning 
procedures adhering to established timelines, planning 
committees with wide representation, strategies and 
institutional priorities based on internal audits and 
environmental scans. Yet these are typically works of fiction. 
The reality bears little resemblance to what is portrayed in the 
pretty shelf documents . Individual offices go about their 
business often unaware of a collegewide plan, and top-level 
decisions are made within the president's immediate staff. The 
"planning process" is largely irrelevant, indulged in by top 
management only because of its public relations value. 

In contrast, the bureaucratic plan is pervasive. Its main 
function is control, and individual offices are required to 
participate as part of the budget-building process. Specific 
staffing and budget requests must be tied to unit goals and 
objectives, which must be linked to institution-wide priorities. 
Rather than the few glossy pages typical of a public relations 
plan, the bureaucratic plan may be incorporated in a thick 
budget document containing detailed goals and budgets for each 
campus unit. While the existence of a bureaucratic plan can 
also serve the legitimation goals of a public relations plan, 
the internal dynamics are totally different. Everyone is aware 
of the plan, and much time and effort are devoted to it. 

Despite their differences, both public relations plans and 
bureaucratic plans typically support the status quo. Action 
plans are different. To facilitate the accomplishment of a 
major presidential or governing board goal, such as a change in 
campus mission, major reorganization, substantial resource 
reallocation, or other fundamental change, a formal planning 
process may be invoked. What sets action plans apart is an 
awareness across the campus that real changes will result, that 
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meaningful decisions will be made, and that the formal process 
at the very least provides an avenue for input. The planning 
process is a tool used to help move the institution in a new 
direction, and is usually taken seriously by the participants. 
Almost by definition, action plans will be infrequent, as 
fundamental change is a rare event. 

Most observers of higher education can recognize and 
probably cite examples of public relations plans, bureaucratic 
plans, and action plans. The concept of a "shadow plan 11 is new. 
Shadow plans are not adopted formally, do not appear in planning 
documents, and in fact are not discussed explicitly on campus. 
They consist of reform ideas that have surfaced in the past but 
have not been enacted. A shadow plan is different from an 
acknowledged contingency plan. To the extent that potential 
challenges can be anticipated, contingency actions can be 
incorporated into any of the plan types. What distinguishes a 
shadow plan is that it remains hidden in the minds of key 
policymakers. Indeed, calling it a plan overstates its 
coherence, in that it represents a collection of ideas and 
reforms held individually by decisionmakers, some of whom are 
more aware and devoted to certain reforms than others. 
Typically the reforms have been discussed, debated, and 
seemingly discarded, in the past. Institutional inertia or 
outright opposition prevented them from being enacted. But key 
players remained committed to them, and their eventual 
implementation only awaited the right opportunity. 

A budget crisis can be the catalyst unleashing ideas hiding 
in the shadows, especially if a plausible argument can be made 
that their implementation will have a positive fiscal impact. 
The three cases described above are examples, and raise several 
questions. Are there ideas "hiding in the shadows" of most 
institutions? If they have not been adopted during normal 
times, does it make sense to adopt them in a crisis? Should a 
deliberate effort be made to bring these ideas to light, to 
assess their impact in a non-crisis atmosphere? Do public 
planning procedures and documents have meaning if shadow plans 
exist? 
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Outreach Programs and 
Their Varying Impact on Key Market Segments 



David J. Costello, PhJD. 
Dean of Enrollment Planning 
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Abstract: 

The outreach efforts of many institutions are extensive and, more importantly, expensive. 
With each passing year, the recruitment effort among colleges for qualified students has increased. 
What has not normally maintained pace with this increased effort is the management and assessment 
of specific outreach programs. This paper analyzes fiow various admission outreach programs impact 
key market segments (e.g., quality of applicant, housing applicant, interviewed applicant). 
Recruitment data from a two-year private institution indicate that outreach programs have varying 
impact on key institutional market segments. 

Introduction: 

There is a story about a customer at a bar commenting on how tasty his beer was. 
His friend, a rather melancholic marketing person, lamented that it was advertising, not 
beer, that he was drinking. In todays competitive educational marketplace, it takes a 
hidden talent at times to discern the real difference between the product of education and 
the marketing of education. 

The marketing and the selling of higher education have never been more overt and 
intense. Outcome measures, marketing niches, segmentation analysis, revenue full-time 
equivalents (RFE's) - the adoption of these terms reveals that the world of higher education 
is moving quickly into the marketing world of Wall Street. This new world — one full of 
consultants, advertising agencies and a host of media outlets — does not come cheaply The 
■underlying premise is that as the competition for students intensifies, the pressure on 
institutions to spend mo A *e money on advertising and marketing increases. 

Willing participants in the ever-increasing competition for students quickly discover 
the apparent contradiction between a cost sensitive market (e.g., students) and a more 
expensive recruitment effort. Furthermore, it is becoming clearer that, in many instances, 
the marketing of the message is clearly muddling the true image of the product. In the long 
run, the increasing dissonance between students' perceptions of the marketing message and 
the actual product delivered needs to be fully addressed by enrollment managers. Yet, 
given the saturation of the marketplace with educational information, enrollment managers 
need to study possible ways to control both the message and the cost of its recruitment 
efforts. 
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At this institution surveys reveal that the costs of education play the most salient role in 
the students 1 enrollment decision. Despite this fact, tuition costs continue to spiral as 
institutions spend increasingly more money in an attempt to remain a viable enrollment 
option for students. For a tuition-dependent institution, the implicit assumption is that 
students with the ability to pay will be, in part, subsidizing their less financially-abled 
counterparts. 

The increased tuition costs underscore the relationship between the ever-increasing 
revenues needed to sustain an institution and the institution's dependency on enrollments 
(as well as the ever-increasing dependency an institution must place on marketing to 
recruit prospective students). It is not the intent of this paper to argue whether or not an 
institution should partake in these marketing endeavors — most institutions cannot afford 
not to participate. The intent of this paper is to show how marketing and recruitment 
efforts is measured in terms of effectiveness and how these measures lead to informed 
enrollment management/marketing decisions. 



Methodology: 

The Setting 

The setting of this study is a two year institution situated in a suburban location 
three miles outside of a large northeastern city. With its career-orientation, the College 
attracts students from the New England region along with relatively healthy enrollments 
from students living in New York and New Jersey. A typical year will see the College 
attract 2,000 applications and enroll approximately 500 students. The College has a mix of 
commuter and resident students with approximately 35% of its students choosing to reside 
in on-campus housing. With rising costs and increased competition with four year 
institutions for traditionally two-year type students, the goal of the College is to expand its 
housing base while maintaining steady new enrollments. 

The Sources of Inquiry 

The study explores the influence of four major sources of inquiry on the enrollment 
decision of 1,123 accepted freshman students. As with most institutions, the College 
employs a variety of techniques in order to generate prospective student interest. From 
planning guides and television commercials to direct mail offerings and on-campus events, 
the initial point of contact with the student is the primary subject of this analysis. For 
analytical purposes, the 80+ inquiry sources are collapsed into four mutually exclusive 
categories. (Listed beneath each major category are examples of specific inquiry sources 
that help constitute the inquiry type.) 

Student Initiated f walk-in, telephone, student referral, etc.); 
Admission Outreach (college fair, high school visits, classroom presentations, etc.); 
General Mailings I Publications (planning guides, print & television media, etc.); and 
■* Direct Mail (Student Search, American Student List, NRCCUA, etc.). 

These inquiry types are major expenditure areas in the admission recruitment effort. More 
importantly, each inquiry type can be measured in terms of its recruitment impact and its 
cost effectiveness. 
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The Key Markets 

It is important to understand that specific inquiry types may have a varying impact 
based on specific market segments. For example, one inquiry source may be quite effective 
in generating application activity among students interested in living in on-campus 
housing. This same inquiry source, however, may not be effective in recruiting a high 
quality applicant. It is important to understand how specific market segments respond to a 
particular inquiry type. 

This study investigates six key market segments. They include: 

Gender (male and female), 

Admission Rating (low, moderate and high), 
n * Application Type (Interview or Mail-In), 
n * Housing Status (Commuter or Resident), 

Application Date (Prior 1/93, 1-2/93, 3-4/93, 5-7/93 and 8-9/93), and 

Enrollment Status (Matriculant and Non-Matriculant). 

It should not be surprising to note that the College's marketing publications and 
overall admission effort are geared toward addressing these key markets. 

The Enrollment Pattern 

Chart 1, on the following page, displays the numeric breakdown of each segment by 
their matriculation status. As noted, men had a 14% higher enrollment yield rate as 
compared to women. Similarly, commuter students, students who interviewed with an 
admission counselor on campus, and students with moderate admission ratings tended to 
enroll at a higher rate compared with their counterparts. 

Similarly, Chart 2 details the application pattern for these students. As noted, the 
conversion rate is significantly higher for students applying on May 1st or later. This chart 
underscores the wide variations in enrollment yields based on the timing of the application. 
The data clearly reveal that the students deemed most desirable by the institution are ones 
that apply during low conversion periods. 
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Chart 2 

Application Date by Matriculation Status 
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The Method 



This study relies on a statistical technique used primarily in non-educational market 
settings. CHAID - a statistical tool that utilizes a chi-square technique to automatically 
differentiate among different groups on a variety of nominal/ordinal data elements. The 
basic principle of CHAID is that it analyzes individual relationships and performs a 
goodness of fit test to determine whether or not the relationship is significant. It then looks 
at a hierarchy in terms of which independent variable best differentiates between a 
dichotomous dependent variable (e.g., matriculation status). To this end, the dependent 
variable in this analysis is the students' matriculation status. Pour separate CHAID 
analyses are conducted with each analysis focusing on a specific inquiry type. Gender, 
housing status, admission rating, interview status, and application date are used as 
independent predictors of a student's matriculation status. 

Data Analysis: 

Chart 3 begins the process of analyzing enrollment yields by inquiry type. Clearly 
enrollment yields vary by as much as 20 percent depending upon the inquiry type (Student 
Initiated Inquiry 51%; Direct Mail Inquiry 31%). This information does imply that, all 
things being equal, specific inquiries can impact enrollment yields. 

Charts 
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Chart 4 presents the results of the first CHAID analysis. This analysis provides insight 
into the independent determinants that influenced the enrollment decision. As noted, this 
analysis included all 1,123 accepted applicants. This chart shows that the overall accepted 
applicant to enrolled student yield rate is 43%. Of the five independent variables used in 
this study, the one that best differentiates students based on their matriculation status is 
the date in which they applied for admission. Of the 1,123 accepted for admission, 568 
applied for admission prior to January, 1993. Among this group of students, only 33% 
enrolled at the College. However, when students applied between May 1st and July 30th 
(N=202), 67% chose to enroll at the College. 
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CHAID is a very useful tool when looking at the various paths that are created. For 
example, noted above was the fact that 568 students applied to the College prior to January, 
1993. The data show that only 33% of these students chose to enroll. However, among 
these students, 265 were interested in commuting to the College. Among this subgroup of 



1 There are three aspects contained In each CHAID box. First Is the name of the attribute. Second Is the number 

of students In this category, while third Is the percentage of this group that actually enrolled at the Institution. For 
example, the top box In Chart 4 contains the application date attribute (Applied 8-9/93). Among this group of 56 students 
(N=55), 86% enrolled at the College (enrollment yield). 
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students, 46% chose to enroll, while only 21% of the students interested in living on campus 
enrolled. Clearly, among this large group of applicants, the College is viewed as relatively 
attractive by commuter students (the 46% is better than the overall rate of 43%) but less 
attractive to potential residence hall students. The College may need to investigate why its 
yield rate among this later group of appli cants is low. 

A second path worth exploring is among students who applied for admission between 
May and July, 1993. As noted in Chart 4, 67% of these 202 applicants enrolled at the 
College. Moreover, amon? the students who interviewed on the campus versus mailing in 
their application, 83% chose to enroll. Of the 105 students who chose to mail-in their 
application only 51% enrolled (yet, if the mail-in applicant was a commuter student, there is 
a 65% s/he enrolled). Once again, this path shows that if the institution can have personal 
contact with the student (the interview process) the chance of that student enrolling at the 
College increases dramatically. 

Chart 5 presents the CHAID results when focusing specifically on Student Initiated 
inquiries. As found in Chart 4, the best predictor of matriculation is the date the student 
applied for admission. Of the 427 accepted applicants in this category, 50% chose to enroll. 
Interestingly, among these students, the enrollment yields differed significantly among 
those who applied prior to May 1st and those who applied on May 1st or later. Of the 311 
students who applied prior to May 1st, 41% enrolled. This contrasts significantly with the 
116 students who applied after the May 1st date. Among this group of students 76% of 
them chose to enroll at the College. 

Charts 
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Following the path for 311 early applicants, the interview process comes into play. 
Of the 156 students who interviewed at the College, 51% enrolled (This percentage 
increases to 64% if the student applied between January and April, 1993). Among the 116 
late applicants (applied between May and September, 1993), 69 interviewed with the 
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College and 88% enrolled. The enrollment yield climbs to 97% if the interviewed student 
was male. 

Chart 6 presents the results of the CHAID analysis performed on 285 accepted 
applicants who inquired to the College via an admission outreach effort. As noted 
previously, the application date is the best determinant of a student's matriculation status. 
Overall, 38% of accepted applicants who were recruited directly by an admission outreach 
program, enrolled at the College. 

Chart 6 

CHAID Analysis: Factors Influencing Enrollment Yields 
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Given the fact that most admission road work is done in the fall and early spring 
recruitment periods, it is not surprising that the vast majority of these students inquired to 
the College prior to May 1st (224 students). Among these students, only 29% chose to 
enroll More importantly, if the accepted applicant was interested in living on campus, the 
enrollment yield rate drops to 18% (136 students fall into this category). Yet, 38% of those 
students interested in residence hall living enrolled if they came to campus and interviewed 
with an admission counselor. 



Chaxl 7, on the following page, details the results of a CHAID analysis performed on 
the 190 accepted applicants who inquired to the College via a general mailing or other non- 
personal medium (e.g., television, planning guides, etc.). As noted, 47% of these accepted 
applicants enrolled at the College. 

It is the housing variable that serves to best differentiate this students' 
matriculation status. Of the 190 students, 110 stated that they were applying to the College 
as a commuter. Among this selected group of students, 5?% chose to enroll. In contrast, of 
the 80 students who stated that they were interested in living in on-campus housing, only 
30% chose to enroll. This enrollment yield rate increases to 48% if the student is 
interviewed on campus by an admission counselor. 
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CHAID Analysis: Factors Influencing Enrollment Yields 



A Focus on General Mailing Inquiries 



i 
i 
i 
i 
i 
i 
i 
i 
i 

i 
i 

i 

r • 

'eric 




N=190 





Lastly, Chart 8 details the findiags from a CHAID analysis performed on the 200 
accepted applicants who first inquired to the College via a Direct Mail As noted, 31% of 
these students chose to enroll at the College. 01 these students, 60 students interviewed at 
the College and 57% enrolled. Of the 140 mail-in applicants only 19% chose to enroll. Yet, 
if these mail-in applicants were commuter students, 38% enrolled. 

Charts 

CHAID Analysis: Factors Influencing Enrollment Yields 
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Implications and Conclusion: 



The five CHAID analyses present in this study reveal that if the initial contact is 
personal (Student Initiated and/or Admission Outreach) the best predictor of enrollment is 
the date of application. However, when the initial contact is less personal (General 
Mailings/Other Media and/or Direct Mail) local knowledge of the institution or personal 
contact are the primary ways to increase enrollments. 

These findings have some significant implications. More than 50% of the College's 
total enrolled pool applied May 1st or later (see Chart 2 for this information). Yet, more 
than 25% of all its accepted applicants apply prior to January. As with most institutions, 
the College expends nearly 75% of its variable recruitment budget during the initial stages 
of the recruitment process (e.g., publications, travel, direct mail campaigns, etc., are 
primarily fall activities) 

What is also clear, not surprisingly, is that local knowledge of the College is a large 
determinant in the enrollment decision. This piece of information is somewhat an ally of 
television and other media advertising where a broad-based exposure may have a residual 
effect on the student marketplace. For example, a student may make a telephone inquiry 
(designated in this study as a Student Initiated inquiry) based on the osmosis effect of 
television advertising. 

Lastly, the CHAID analyses clearly showed the importance of the interview process. 
Based on this information, the institution needs to analyze how it encourages students to 
visit the campus. It needs to understand the potential blocking mechanisms that may stop 
an out-of-state student from visiting the campus. 

In sum, based on these and other analyses the College will review how it outlays 
monies to specific inquiry generation. Furthermore, given that certain inquiry generators 
will remain (e.g., Admission Outreach), a review of both the messages being imparted and 
when the message is being delivered will be conducted. 
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Analytical lechniques for Studying Student Retention 



Anne Marie Delaney 
Director of Program Research, School of Education 
Boston College 

Introduction 

The purpose of this paper is to discuss data requirements for a longitudinal retention 
file and to present the design and analytical techniques employed in retention studies that 
have produced information responsive to the questions and concerns of university 
administrators. Proposed enhancements to these studies to include socially relevant variables 
are also discussed. The three major topics addressed in this paper include: 

- Data Requirements for Designing a Longitudinal Retention File, 

- Logic for Developing Computer Retention Programs, and 

- Techniques for Analyzing and Predicting Retention 

Accurate and complete information is critically important in the design of retention 
programs. As Glover and Wilcox (1992) observe, the ability to diagnose student retention 
problems and to design effective retention strategies requires the establishment of a 
comprehensive multi-year student data base that incorporates both student records and 
student survey data. 

Background 

Results from previous retention research studies provide a rich resource of ideas for 
the development of a longitudinal retention data base. These studies have investigated the 
effects of various student characteristics and institutional variables on retention rates at 
higher education institutions. Student demographic characteristics, academic qualifications, 
personality traits, college housing status, student-faculty relationships and financial factors 
are among the major categories of variables examined (Pantages, 1978). 

Findings from several of these studies document that academic and social integration 
contribute substantially and significantly to retention (Pascarella & Terenzini, 1983; Terenzini 
& Pascarella et al., 1985). In these studies, academic integration is defined by cumulative 
freshman year G.P.A.'s, a scale measuring perceived level of intellectual development, 
student perception of faculty concern for student development, and the frequency of 
students' out-of-class contact with faculty for some academic or course-related matter. Social 
integration involves the average number of hours per week spent in organized extracurricular 
activities and the extent and quality of students' interaction with peers (Terenzini and 
Pascarella et al. , 1985). Further empirical studies also show that informal student-faculty 
interaction is either the most or second most important single variable in integration 
(Terenzini & Pascarella, 1980). 

Other retention researchers also emphasize the strong link between social and 
intellectual integration and retention. In their book entitled Increasing Student Retention . 
Lee Noel and associates (1985) observe that social and intellectual integration reinforce 
institutional commitment and enhance persistence. Academic integration appears to be 
somewhat more important than social integration for academically superior students. Finally, 
frequent contact with faculty outside the classroom appears to be one of the most important 
components of integration affecting persistence. 
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Some of the variables potentially relevant to retention, such as demographic 
characteristics, academic performance, housing status and financial profiles, may be stored 
on an administrative data base. Other data, particularly information related to social 
integration, may be obtained through student surveys. Examples include: Student 
Satisfaction with Campus Life, Student Involvement in Campus Activities, Student 
Leadership Roles, Student Service Activities, and Student Relationships with Faculty, Staff 
and Other Students. These data can then be merged with the information contained in the 
type of administrative data base discussed in this paper. 

Data Requirements for a Longitudinal Retention File 

A longitudinal retention file may need to serve many purposes, including monitoring 
the progress of students from entrance to completion, documenting graduation rates for 
classes as a whole and for subgroups within the classes, determining the time of attrition, and 
identifying student characteristics related to graduation and attrition. These many purposes 
need to be taken into account in determining the elements of the file and in planning the 
procedures for creating and updating the information for each entering cohort. 

The following proposed list of data elements has been developed with the previously 
stated purposes in mind. The suggested comprehensive data base is intended to provide a 
basis for documenting retention and for conducting research on reasons why certain groups 
of students complete their education and graduate and other groups of students leave prior to 
graduation. Most of the data elements related to understanding the factors associated with 
retention were selected on the basis of findings from previous retention research studies. 



Pro posed Data Elements for a Longitudinal Retention File 



Personal Background Information 



Admission Information 



I.D. 

Name - Last, First 
Birth Date 
Gender 
Citizenship 

Racial/Ethnic Background 
Family Income 
Parents' Education 



High School 

Quality Ranking of High School 
SAT Scores 
High School Rank 

Size of High School Graduating Class 

Admission Rating 

Entrance Status 

Early Notification 

Entrance College 

Entrance Major 

Alumni Relation 

High School Achievements and Honors 



Academic Semester Profile 



Academic Summary Data 



Present Term 
Present Status 
Present College 
Present Major 
Term Credit Hours 
Cumulative Credit Hours 
Cumulative G.P.A. 
Major Cumulative Average 



Cumulative G.P.A 
Total Credits 
Final Major 
Degree 
Final Status 
Graduating Class 
Date of Graduation 
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Financial Aid Information 



Applicant Status 
Family Income 
Need 

Parents' Contribution 



Financial Aid Awards by Semester 
External Scholarships and Grants 
Institutional Scholarships & Grants 
Loans 

Work Study Awards 

Other Work 

Other Financial Aid 



Procedural Guidelines for Creating and Maintaining a Longitudinal Retention File 

A separate raw data file, containing the previously specified elements, should be 
created for each entering freshman class. A production schedule needs to be established and 
followed to create and update the files each semester over a six year period. It would be 
advisable to read the files each semester as a quality control measure in order to ensure that 
the system is working as planned. 

Experience with active administrative data bases prompts the recommendation to 
review and assess the accuracy and completeness of the data before producing retention 
reports or conducting retention research. The following section outlines procedures the 
author established during the course of the first retention study. In the first year of producing 
a retention report, the author performed most of these tasks. In subsequent years, student 
research assistants followed these guidelines to complete the tasks. Completion of these 
tasks contributed significantly to producing an accurate and reliable data base for conducting 
retention research. 

Procedures for Transforming Administrative Data into Retention Research Data Base 

1. Verify the total count by selecting cases based on Present Status and Entrance 

College. Verify the number with administrative records. 

2. Obtain a Breakdown by Final Leave Status. Evaluate results using trend 

data as a guide. 

3. Verify Graduation Status for all listed Graduates. 

4. Look for additional Graduates. 

A. Check all student with blank Final Leave Status. 

B. Check status of all non-graduates with the final transcripts. 

5. Identify and Delete Duplicate Records. 

A. Run Frequencies of Social Security Number or I.D. 

B. Determine which record is accurate; delete other. 

6. Identify potential missing data. 

A. Verify completeness of academic information for Graduates. Compare 

existing semester academic information with number of semesters 
graduates would be expected to attend. 

B. Check other key variables - Present Status and Present College. 
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Once the longitudinal raw data file is complete, programs, possibly using SPSS, 
might be written to analyze the data. The logic of sample programs is outlined in the 
following section. 

Lo gic for Developing Computer Retention Programs 

This section presents the logic used, with SPSS, to create computed variables from 
existing data in the retention research data base. These computed variables are relevant to 
tracking students from entrance to graduation or departure and to tracking students within the 
same university, from one undergraduate school to another. The programs identify which 
students entered and persisted in the same school and which students changed schools after 
entrance; the final semester both for graduates and non-graduates; and the final leave statuses 
by Entrance and Final Schools. This information derived from these programs provide 
comparative data for evaluating variations in persistence rates within the same school and 
overall college graduation rates by undergraduate schools. The following section provides an 
overview of the logic for the computer programs. More detailed information could be 
obtained from the author. 

Program Logic for Documenting Graduation Rates by Entrance and Final Schools 

The following information outlines the major variables computed and the sequence 
followed in computing these variables to produce graduation rates by students' Entrance and 
Final School. The program monitors progress over six years, from the first through the 
eighteenth semester. 

Computed Variables 

First School (FIRSTSC) 
First Semester (FIRSTSEM) 

School Change (SCCHANGE) - indicates if a student transferred within 

the university 

Last School (LASTSC) 
Last Semester (LASTSEM) 

Use 'Do Repeat* to determine Last School and Last Semester 

A= Schools from Present College(PCl) to Present College 

(PC 18) 

B=Count 1 to 18 

Last School Graduation Status (LSCSTAT) 

IF (LASTSC EQ'01' AND CLS EQ ' G ')LSCSTAT=1 
IF (LASTSC EQ ' 01' AND CLS NE ' G ')LSCSTAT=2 

These statements provide a breakdown of Graduates and Non-Graduates 
for College ' 01 \ 

Run : Crosstabs: PCI by LSCSTAT to produce Graduation rates by Entrance 

and Final Schools. 
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Program for Documenting S gggifig Final Leave Statuses bv Entrance 
and Final Schools 

The second retention program provides more detailed information with specific final 
leave statuses by students' Entrance and Final Schools. The two variables used in this 
program are: Final Leave Status (CLS) and First and Last School (FIRLASSC). The former 
variable, CLS, is in the original data base. The Final Leave Statuses include Graduate, Leave 
of Absence, Mandatory Withdrawal, and Voluntary Withdrawal. The latter variable, 
FIRLASSC, is a computed variable. Guidelines for computing this variable follow: 

Compute one variable (FIRLASSC) to include all possible combinations of Entrance 
and Final Schools. Sample statements to compute categories for this variable, with 
four undergraduate schools, follow: 

IF (PCI EQ ' Or AND LASTSC EQ ' 01' ) FIRLASSC=1 

IF (PCI EQ ' 08' AND LASTSC EQ 1 09' ) FIRLASSC=16 

Run: Crosstabs: CLS by FIRLASSC to produce Final Leave Statuses by 

Entrance and Final Schools. 

Pro gram for Documenting Final Semester bv Final Leave Status 

The third retention program identifies the final semester for graduates and non- 
graduates, broken down by final leave status such as Voluntary Withdrawal, Leave of 
Absence, Mandatory Withdrawal. This information is very important in determining when 
students graduate and when non-graduates decide to leave the college. The two variables 
used in this program are Final Leave Status (CLS) which is in the original data base and Last 
Semester (LASTSEM) which is a computed variable. The information is produced by 
running a Crosstabs Program: LASTSEM by CLS. 

Techniques for Analyzing and Predicting Retention 

As noted earlier, the proposed longitudinal retention file, with the original and 
computed variables, is intended to serve as a basis for documenting retention rates, 
monitoring students' progress from entrance to graduation or departure, and identifying 
student characteristics related to variation in graduation rates. 

Initially, bivariate analyses might be used to determine which variables appear to be 
significantly related to retention; these variables might then be included as predictors or 
independent variables in multivariate procedures designed to predict retention as the outcome 
or dependent variable. An appropriate technique for predicting retention is Discriminant 
Analysis - a regression approach with a dichotomous variable as the dependent variable. 

In a recently completed longitudinal retention study, the following variables were 
included as independent variables in a discriminant analysis of retention among one major 
cohort of students: Freshman Cumulative Average, Resident Status, Admission Rating, High 
School Rank, and SAT Scores. Results from this Discriminant Analysis are presented in 
Table 1. 




TABLE 1 

DISCRIMINANT ANALYSIS: PREDICTING STUDENT RETENTION 



Measures Structure Coefficients 

Freshman Cumulative Average .92 

Resident Status (2 = Resident; 1 = Commuter) .30 

Admission Rating (1 = High; 10 = Low) -.27 

High School Rank ' -.25 

SAT Verbal Score -.02 

SAT Math Score .08 



Classification Results 



Graduates 
Non-Graduates 



N 
6321 

874 



Centroids 
.11 

-.81 



Predicted Group Membership 
Graduates Non-Graduates 



6287 
(99.5 %) 

755 
(86.4 %) 



34 
(0.5 %) 

119 
(13.6%) 



Percent of Cases Correcdy Classified: 89.03 % 



Results From Function Test 

Rcan = .28 x 2 = 542.21 df = 5 p < .001 

As shown in Table 1 , the Freshman Cumulative Average correlates very strongly, .92, 
with the function predicting students' graduation. Though to a substantially lesser degree, 
Resident Status, Admission Rating, and High School Rank also correlate positively. The 
remaining correlations for the variables included in the function are very small - negative for 
the SAT Verbal Score and positive for the SAT Math Score. Overall, the model accurately 
predicts 89 percent of the cases. The high correlation among the existing variables is a 
limitation associated with this analysis. 

While it is of interest to observe that freshman cumulative average is the strongest 
predictor of retention, it is perhaps even more critical to determine what factors are likely to 
predict freshman academic performance. This information may serve to identify students at 
risk of dropping out prior to graduation. Further, data from this retention study show that the 
majority of students leave either at the end of the first or second year of college. 

In the longitudinal retention study reported in this paper, regression analyses were 
conducted to identify predictors of freshman cumulative average. In presenting these results, 
it is important to point out that the selection of independent variables was limited by the 
available data. The author believes that greater insight into the causes of attrition could be 
gained with the inclusion of more variables, particularly those related to social integration. 

Results from multiple regression predicting freshman cumulative average from a set 
of admission predictors are presented in Table 2. 
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As shown in Table 2, the Admission Rating explains 24 percent of the variance in 
these students' freshman academic performance. With this variable in the equation, the SAT 
Score and High School Rank each explain an additional 1 percent of the variance. As noted 
previously, since the independent variables are highly correlated, once the first variable is in 
the equation, the other variables can contribute little to explaining the shared variance. The 
amount of unexplained variance suggests the need for additional independent variables. As 
discussed earlier in this paper, it may be very useful to expand the data base to include more 
data concerning students' psychological adjustment and social integration during their early 
college experience. Interviews with selected students suggest that the social-psychological 
dimension may be the most critical in determining whether or not at risk students persist to 
graduation. 
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Designing Alumni Research for Assessment and Plannin g 



Anne Marie Delaney 
Director of Program Research, School of Education 
Boston College 

Introduction 

In recent years, the role of alumni research has assumed particular significance given 
the increased emphasis on outcomes assessment (Pike, 1990). Further, as Williford and 
Moden (1989) observe, a unique feature of alumni surveys, compared with surveys of 
enrolled students, is the capability of documenting students' assessment of the quality of their 
educational experience tempered by their experiences since graduation. Results from alumni 
research also provide a basis for examining the relationship between satisfaction with college 
and academic major (Richardson, 1993), perception of learning during college (Pike, 1993), 
and subsequent work experience (Pike, 1993a). 

The purpose of this paper is to present the design, implementation strategies, 
analytical techniques and significant results from a recently completed undergraduate alumni 
study administered to the 1987 to 1991 graduating classes from the Boston College School of 
Education. While much of the content of the original study addressed issues related to 
teacher education and human development programs, the paper will show how the 
methodology can be applied to different academic disciplines and professional schools. The 
paper will demonstrate how alumni research can be designed and utilized to meet various 
institutional research purposes including Assessment, Program Evaluation, and Program 
Planning. 

Design Considerations 

The population surveyed in this alumni survey included five graduating classes - from 
1987 through 1991 yielding a total of 800 alumni. This population provided an opportunity to 
document trends in the relationships between post-graduate experience and perception of the 
value of the undergraduate education and satisfaction with undergraduate education, and to 
relate changes in graduates' feedback to changes made in the curriculum during this time 
period. 

Population 

The issue of size is critical to the ability to analyze variations in the data and to make 
inferences to the population. Statistical formulae, taking into account levels of statistical 
significance and power, may be used to determine sufficient sample sizes to support tests of 
significance and inferences to the population. 

It is also advisable to determine, prior to conducting the study, what subgroup 
analyses will be conducted, and to estimate not only the overall response rate but also the 
response rates for individual classes for which inferences might be made. In this study, the 
overall response rate was 58.76 percent and the response rates for individual academic 
programs ranged from 52 to 62 percent. 
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Survey Instrument 



The survey instrument in this alumni study was, in part, the product of a faculty 
committee. My involvement occurred after most of the questions had been determined. 
Therefore, my role focused primarily on refining the questions and designing the 
questionnaire format. In this presentation, I will address general principles that ought to be 
considered in the design of alumni surveys, and, where appropriate, utilize this alumni survey 
for illustrative purposes. 

The following outline and questions may serve as a useful guide in the development 
of alumni survey instruments . 

1. W hat goals, components, or characteristics of the program do vou want to evaluate ? 

Quality of Academic Life 

Goals: Review Mission Statements, Policy Statements, and 

College Catalogues 

C Qmponents: Professors - Teaching, Advising 

Classes - Content and Degree of Challenge 
Class Discussions - Value 
Assignments - Relevance and Challenge 
Internships - Quality of Experience and Supervision 

Pro gram Characteristics : Balance of Theory and Practice 

Up-to-Date Approaches 

Quality of Student Life 

Intellectual Milieu: Intellectual environment of the campus 

Cultural Life: Cultural opportunities available to students 

Social Opportunities: Opportunities for meaningful relationships & activities 



2. What information would be relevant regarding post-graduate employment? 

Employment Seeking Experience: Time, Challenge, Positions Sought, Efforts 

Types of Positions Held: Titles and Length of Time 

Challenges Encountered 

Strategies Employed to Deal with Challenges 

Challenges for which They Could Have Been Better Prepared 
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3. What information about respondents would be potentiall y re levant to analyzing 
variation in their evaluation of their education and their post-graduate 
employment experience ? 

Undergraduate Prof ile: 

Academic - Hours of Study, Investment in Learning, Grades, Honors 
Internship Experiences - Quality and Relevance 
Student Life - Leadership Roles and Involvement in Activities 
Personal Commitment - Family and Work 

Post-Graduate 

Graduate School - Institution, Major, Degree and Year of Completion 
Affiliation with the Institution since Graduation 
Employers - Obtain names of first job and current employers 



Boston College School of Education Alumni Survey 

This survey covered three major areas: Graduates 1 Evaluation of their Undergraduate 
Program, Graduates' Post-Graduate Educational and Employment Experience, and Selected 
Background Information. The following outline indicates the principal components in each 
of these areas: 

Evaluation of their Undergraduate Program 

What was their primary expectation? 

Flow challenging were aspects of their undergraduate education - 
classes and field experiences? 

How satisfied were they with each of the following: practica, supervision, 
classes, and university resources? 

How much did their education contribute to their growth in different areas: 
communication, critical thinking, multi-cultural awareness and 
value awareness? 
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Boston College School .f Education Ah rmni.Suryev (continued) 
Post-Graduate Employ - 1 $ e it 

Employment Seeking ) xperience: Wl a t kind of job were they seeking? 

How difficult was it to find employment? 
Weu'; they unemployed after graduation and 

for how long? 
Were they willing to relocate for employment? 

Employment Experien ;e: Types oi' Positions Held Since Graduation 

Professional Challenges Encountered 

Examples for Teaching follow: 

Developing a Rapport with Students 
Evaluating Students' Performance 
Evaluating Own Teaching 
Mainstreaming 
Maintaining Discipline 
Motivating Students 
Planning Class Instruction 
Relating to Administrators 
Relating to Students' Parents 
Teaching Students of Different Ability 

Levels in the Same Class 
Teaching Students from Different Socio- 

Economic Backgrounds 

Problems that Might Have Been Better 
Addressed 

Background Information 

Undergraduate Profile: Commuter vs. Resident Status 

Regular vs. Transfer Status 
Undergraduate Major 
Year of Graduation 

Post-Graduate Profile Further Education 

Annual Income 

Implementation Strategies 

With some modifications, implementation strategies for this survey generally 
followed the procedures outlined in Dillman's (1978) book, Mail and Telepho ne Surveys. 
Data collection efforts included a reminder post-card sent one week after the initial mailing 
and two subsequent follow-up mailings that included the complete survey package. A cover 
letter from the Dean of the School of Education was sent with each of the complete survey 
mailings. This letter stressed the significance of the project to the School, the importance of 
each respondent's contribution, an assurance of confidentiality, an offer of results and a 
contact for questions. While assuring confidentiality, the surveys included an I.D. ; this was 
essential for monitoring returns. It also provides a means of merging the survey data with 
information from administrative files. 
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Presentation of Results: Administrative Reports and Research Possibilities 



Administrative Report 

Results from this Alumni Survey were initially presented to the Dean in a 
comprehensive report of the overall survey results. The contents of the report included a 
comprehensive analysis of alumni responses broken down by year of graduation and by 
School of Education major. Graduates' evaluation of their undergraduate education related to 
a number of dimensions including their perception of the degree of challenge offered by their 
courses and field experiences; their level of satisfaction with courses, student teaching, 
supervision and university resources; and their assessment of the degree to which their 
undergraduate education contributed to their professional growth in developing various 
personal and intellectual skills such as self-understanding and critical thinking. 

The Boston College School cf Education Alumni Survey Report also provided an 
extensive employment profile documenting current employment status, satisfaction with 
current employment, geographic region of current employment and approximate net personal 
income last year. Information on graduates' employment seeking experience was also 
presented including the kinds of employment sought; difficulty in finding employment; 
willingness to relocate for employment and a history of employment since graduation. 

The most extensive portion of the employment profile focused on graduates* 
professional teaching experience. This section included graduates' evaluation of their teacher 
preparation program by major, types of teaching positions held; subject areas in which 
graduates were certified to teach and had taught; number of graduates who had taught in their 
area of certification; graduates' perception of work related problems in teaching and their 
recommendations for addressing these problems in the undergraduate curricula. Graduates' 
non-teaching employment experience was also documented including the reasons why 
graduates are not in a teaching position. 

In addition to the overall results, comparative data were also presented by Academic 
Program. This was particularly helpful in evaluating graduates' feedback on then- 
professional teaching experience and their evaluation of their undergraduate curriculum. 
When asked to identify problems that should have been better addressed in then- 
undergraduate curriculum, graduates as a whole most frequently mentioned Teaching 
Students of Different Ability Levels followed by Maintaining Discipline and Relating to 
Parents. However, as illustrated in Table 1, graduates from different undergraduate majors 
varied to some extent in the ranking of these problems. Special Education majors were quite 
different from the group as a whole; they most frequendy mentioned Relating to Parents, 
followed by Teaching Students of Different Ability Levels, Relating to Administrators and 
Teaching Students of Different Socioeconomic Backgrounds as problems in need of greater 
attention in the curriculum. Secondary Education majors most frequently cited Maintaining 
Discipline followed by Teaching Students of Different Ability Levels and Motivating 
Students. 
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Table 1 

Ranking of Problems Graduates Identified that Should Be Better Addressed 
in their Undergraduate Curriculum 





School of Education Major 










Earlv, 


El em. 


Sec* 


Special 


Human 


Middle 




Problem 


Ed 


Ed. 


Ed 


Ed 


Dev. 


School 


Total 


Teaching Students of 


1 


1 


2 


2 


2 


1 


1 


Diff. Ability Levels 




(73) 


(14) 


(24) 


(10) 


(4) 


(128) 


Maintaining Discipline 


2 


3 


i 


8 


1 


3 


2 




(25) 


(59) 


(21) 


(13) 


(11) 


(2) 


(117) 


Relating to Parents 


3 


2 


4.5 


1 


4 


3 


3 




(23) 


(65) 


(9) 


(25) 


(6) 


(2) 


(111) 


Evaluating Students 


5 


4 


10 


9.5 


5.5 


7 


4 


Performance 


(16) 


(49) 


(3) 


(12) 


(5) 


(1) 


(74) 


Teaching Students from 


4 


8 


4,5 


3,5 


8 


7 


5 


Diff. SES Background 


(17) 


(27) 


(9) 


(16) 


(3) 


(1) 


(61) 


Mainstxeaming 


7 


5 


8 


5.5 


3 


3 


6 




(11) 


(33) 


'6) 


(15) 


(7) 


(2) 


(60) 


Planning Class 


6 


6 


6 


5.5 


8 


7 


7 


Instruction 


(14) 


(29) 


(7) 


(15) 


(3) 


(1) 


(53) 


Motivating Students 


10 


8 


3 


9.5 


10 


7 


8 




(7) 


(27) 


(12) 


(12) 


(2) 


(1) 


(51) 


Relating to 


8.5 


8 


8 


3.5 


8 


7 


9 


Administrators 


(10) 


(27) 


(6) 


(16) 


(3) 


(1) 


(50) 


Evaluating Own 


8.5 


10 


8 


7 


5.5 




10 


Teaching 


(10) 


(26) 


(6) 


(14) 


(5) 




(49) 


Developing Rapport with 


11 


11 


11 


11 






11 


Students 


(2) 


(7) 


(2) 


(4) 






(12) 



Faculty Presentation 

Following the submission of the initial administrative report, results were presented to 
the faculty. This presentation highlighted strengths and areas in need of improvement for the 
School as a whole with some mention of differences by Academic Program. The quantitative 
results were presented primarily by means of graphs. The very favorable response received 
from the faculty may be related to the accessibility of the graphic presentation and to the 
deliberate focus on program strengths and to the presentation of areas in need of 
improvement as constructive possibilities for program enhancements. 
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Academic Program Reports 



Subsequent to the faculty presentation, individual survey reports were prepared for 
each Program Chairperson. These reports contained the responses of alumni who graduated 
from the specific Academic Program. Program Chairs-sons were encouraged to review 
these data and share them with their faculty in ongoing curricula review and revision. 

Determinant of Successful Utilization 

Administrative vision and leadership, combined with a strong commitment to 
planning based on research, are essential determinants of utilization of program research. 
The Boston College School of Education Alumni Survey benefited from the presence of this 
spirit and commitment in the support of the Dean of the School of Education and, with her 
leadership, the support of the Chairpersons and Faculty. 

Other Analyses: Ideas for Further Research 

Chi Square and correlational analyses were conducted to examine the relationship 
between satisfaction with employment, level of income and graduates' evaluation of their 
undergraduate education as a preparation for their career. Bivariate analyses also examined 
the relationship between employment satisfaction and perceived growth achieved through 
undergraduate education. As shown in Table 2, statistically significant relationships were 
found between employment satisfaction and graduates* evaluation of the preparation they 
received for their careers. 

Table 2 

Relationship of Alumni Employment Satisfaction to 
Graduates' Evaluation of Their Undergraduate Preparation 

A. Teacher Education Alumni Satisfaction with Employment 

Evaluation of 



Undergraduate 








Very High 


Total N 


Preparation 


Low 


Moderate 


High 


Poor to Fair 


42.9 


28.8 


22.2 


20.4 


74 


Well 


32.1 


49.2 


52.8 


35.2 


126 


Very Well 


25.0 % 


22.0% 


25.0 % 


44.4 % 


101 


100.0% 


100.0% 


100.0% 


100.0% 




Total 


(28) 


(59) 


(72) 


(142) 


N=301 










X 2 =19.8 


(P < .01) 


R. Non-Teacher Education Alumni Satisfaction with Employment 


Was Program 












Helpful? 


Low 


Moderate 


High 


Very High 


Total N 


No 


57.1 % 


27.0 % 


12.2 % 


14.3 % 


33 


Yes 


42.9 


73.0 


87.8 


85.7 


129 




100.0 % 


100.0 % 


100.0 % 


100.0 % 




Total 


(14) 


(37) 


(41) 


(70) 


(N=162) 










X 2 =16.0 


(P < .01) 
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As illustrated in Table 2, among Teacher Education graduates, 44 percent of those 
who express 'Very High' satisfaction with their employment, compared with only 25 percent 
of those who express 'Low' satisfaction with their employment, report that their 
undergraduate education prepared them 'Very Well' for their career. Similarly, among 
graduates of Non-Teacher Education programs, 86 percent of those who express 'Very High' 
satisfaction, compared with 43 percent of those who express 'Low' employment satisfaction, 
report that their undergraduate program was helpful in preparing them for their present 
employment. 

Among the various areas of perceived growth examined - Communication Skills, 
Critical Thinking, Multicultural Awareness, and Problem Solving - results revealed one 
statistically significant relationship between Employment Satisfaction and growth in 
Communication Skills. Some 50 percent of those who report 'Very High' employment 
satisfaction, compared with 29 percent of those who report 'Low' employment satisfaction, 
report their Communication Skills grew 'Very Much' through their undergraduate education. 

Discussion of Noteworthy Findings 

Results from this alumni survey yielded both quantitative and qualitative data. At the 
conclusion of the survey, respondents were invited to share their perspectives regarding what 
aspects of their undergraduate program they found most helpful and what aspects might be 
improved or strengthened for future students. Four major themes emerged from analyses of 
these qualitative comments: Recommended Changes in the Curriculum, Expansion of the 
Internship Experience, Increased Career Advising, and More Integration of the Human 
Development Major in the School of Education. The ideas offered in the qualitative 
comments were generally consistent with results from the quantitative data. Highlights from 
these findings follow. 

Over eighty percent of the graduates expressed satisfaction with their undergraduate 
education. At the same time, there was variation in the perceived level of challenge in certain 
courses. Comparative data indicated the need for increased challenge in certain courses. 

Perhaps reflecting the influence of their experience since graduation, alumni 
strongly advocated more application of theory to practice during undergraduate education. 

Consistent with the emphasis on practice, alumni applauded the early teaching 
experience afforded through a sophomore year pre-practicum and advocated expanding both 
the time and the level of responsibility associated with existing practicum experiences. 

With a focus on the relationship between education and employment, alumni also 
strongly recommended early, realistic and more intensive career advising for all 
undergraduates. 

Finally, while results from this survey documented some themes common to all 
alumni, the data also revealed variations by academic program in terms of perceived need for 
changes in the curriculum, experience in finding employment, and professional challenges 
encountered in graduates' early careers. This program specific information is currently being 
utilized in ongoing program evaluation and curriculum development at departmental levels 
throughout the School of Education. 
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File transfer without direct Internet access from desktop computer 



DeBinHex 
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Remote file server 
anyplaceJnternet.edu 



1. Connect to campus mainframe via 
modem. 

2. ftp anyplaceanternet.edu 

3. login: your.id 

4. Password: your.password 

5. cd dirname (change to proper 
directory path) 

6. binary (set binary transfer type) 

7. get filename 
8* quit or bye 

9. Invoke mainframe 
telecommunications package 
in binary mode 

10. Invoke desktop 
telecommunications package 
in binary mode 

11. Download file 

12. DeBinHex file 

13. Open file with Lotus or Excel 
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THE INTERNET TOOLBOX (? indicates software untested) 











Electronic Mail 


Eudora 1.4 


Elm 
Unix mail 
Pine 

vi, pico, emacs 


Pegasus ? 
readmail.exe ? 
Nupop ? 
FopMail I 
PC-Eudora ? (TO A) 


Telnet/TN3270 

(use TN3270 for IBM mainframes) 


NCSA Telnet 2.5 
Comet (a tn3270 
tool) 


telnet/tn3270 


telnctd.zip ? 
CUTCP package ? 


FTP (File Transfer Protocol) 

File compression/conversion 

(GlF/JPEG/anim/audio formats) 


Fetch 2.1 

Stuffu family 
many converters 


ftp 

compress/uncompress 
tar 


simpleftp ? 
goftp.com ? 

PKzip/PKunzip 

many converters 


Usenet News 


News watcher 
Nuntius 


rn 
nn 


news.exe? 
snuz.exe 


Information Locators 


Archie client 

waisotauon 

MacWWW 


archie, veronica*, jughead, 
traveller 

World Wide Web 

Sonoma.sh 

SWAIS 

finger, whois, nslookup 
Netfind, Knowbot 


InfoPop 2.4 ? 


"Front-ends" 


GopherApp 
TurboGopher 
Mac Hytelnet 
MacWWW 

iVIvj5alU 


Sonomash (Ltbs.sh) 
Hytelnet 

ITS front end menu 
Lynx (WWW client) 


PC Gopher II ? 
HGopher (Windows) ? 
Hytelnet ? 
libnei.^xe ? 
iNcoA .viosaic 


Dial-in connection 


SLIP 
NetSerial 

AppleTalk Remote Access ? 
PPP (point to point protocol) 


Modem/serial dial-in 


7 



Very Easy Rodent-Oriented Net- wide Index to Computerized Archives. 




81 



105 



r 





ERIC 



To Have and To Hold: On the Meaning of "Retention" 



Thomas B. Flaherty 
Director of Planning and Institutional Research 
Central Connecticut State University 

and 

Jennifer A. Brown 
Director of Institutional Research 
Connecticut State University 

Abstract 

The purpose of this paper is to describe attendance patterns at Central Connecticut State 
University (CCSU), a public, four-year, comprehensive university and to discuss the issues that 
should be addressed in light of this data. Particular attention is given to the implications of the 
extraordinarily varied paths students follow to the baccalaureate. The relevance of retention and 
completion rates as usually conceived, for this type of institution as well as others, will be dis- 
cussed 

Synopsis of Presentation^ 

One of the most discernible trends in higher education during the 1980's and continuing 
into the 1990's has been the accountability movement. The expectation that institutions of high- 
er learning would show evidence that they were delivering services of quality has now become 
part of the culture of higher education. Nearly every college and university now has an assess- 
ment plan, often mandated by a higher authority-the legislature or board of trustees. Usually, 
the plan involves some form of retention or graduation-rate measure. In the nineties, we have 
received instructions from the NCAA and even Congress to calculate and report on our retention 
of undergraduates and the rate at which they graduate. It is taken as a truism that colleges of 
quality keep their students and graduate them within some reasonable period of time, normally 
four years or a little more for four- year institutions. Few have questioned this measure of insti- 
tutional ''goodness;" it just seems obvious. Recently, however, Astin (1993) has raised questions 
about the appropriateness of this idea of retention. 

Some universities and colleges, particularly four-year public comprehensives and commu- 
nity colleges, however, have been undergoing changes in their missions and in the make-up of 
their student bodies that make the whole idea of retention difficult to comprehend. Many stu- 
dents come to these institutions with the intention of taking a few courses to help them personal- 
ly of professionally. Others come with the idea of transferring to another, perhaps more presti- 

; A formal paper containing the materia) presented at the conference was not prepared. Additional details may be obtained 
from the authors. 
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gious or expensive institution. Perhaps most challenging to the notion of the freshman cohort 
moving through to graduation — the "class" — is the fact that many of the four-year comprehen- 
sive colleges and universities take 50% or more of their incoming classes as transfer students. 



Destiny of 
Fall, 1986 Cohort 



30 



First, a look at our retention data as usually reported. The chart in figure 1 provides a sum- 
mary. The data here are not unlike those of other universities in the Connecticut State 
University system and are apparently 
like those reported at other similar insti- 
tutions. About 25% of the freshman 
cohort fails to reappear for the sopho- 
more year. Then lower attrition rates 
prevail until some six years later when 
the percentage of the freshman cohort 
graduating comes in at about 45%. 
Using earlier cohorts, we surmise that 
our eventual graduation rate is about 
50%. 
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Figure I. Retention/attrition of students entering in the 
Fall of 1986 



At Central Connecticut State 
University, we knew that all of these 
things were true, but we also knew that 
retention was a far more complex matter 
than the current schemes for measuring it could accommodate. For this reason, and because we 
needed to know more about the success of students transferring in from the public community 
colleges, a study was conducted. We wanted to find out what the attendance patterns of our 
graduates had been, how many of them had transferred in how much credit from which institu- 
tions and how long it took them to graduate. 

Our student database, while quite sophisticated for most purposes, simply did not contain 
enough detail to "track" our graduates' collegiate experience completely. Thus, we resorted to a 
transcript analysis. We originally hoped to analyze the transcripts of several hundred students to 
determine how they moved through the institution. Because of the complexity of the student's 
attendance patterns, the task turned out to be so time consuming, that we collected complete data 
on just 81 recent graduates. 

In our presentation, we discussed our findings at some length. Generally, we have found 
that the attendance patterns of our students, both transfer and "native", vary beyond anything we 
had imagined. Indeed, to us, the whole definition of "transfer student" has become an increas- 
ingly fuzzy one. We have found that it is difficult to classify students in any but the most trivial 
ways (e.g., started at our university or started elsewhere) because of the frequency of transfers — 
in and out. The opportunity to apply quantitative methods was far more limited than we had 
hoped. The complexity of attendance patterns, individual differences in the number of institu- 
tions attended, and the amount of credit earned before and during enrollment at the receiving 
institution and so on, reduce the utility of quantitative analysis and make case-by-case analysis 
necessary. 
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Starting Dates in Higher Ed. 

(For 19S1 Graduates) 



□ "Native" 
b Transfer 



Some interesting statistics were 
gleaned, however. Of the random sample 
of 1991 grads, 41% had begun at 
CCSU-were native students-and 59% 
were transfer students (the latter wholly 
ignored by traditional retention analysis). 
This split is typical of our entering stu- 
dents. While 87% of the native grads had 
matriculated at CCSU in the Fall of 1985 
or more recently (in the six-year frame), 
only 45% of transfer students offered 
their transfer credits in Fall, 1985 or 
thereafter. The whole sample reflected a 
range of start dates from Spring 1989 
(transfer) to Fall, 1955. Overall 6% of 
the native students and 16% of the transfer students had initial enrollment dates of 1979 or earli- 
er. It is important to note here that these students were persisters, not drop-outs. Figure 2 shows 
the entering dates of all of the students in the study and gives a good idea as to the number of 
years taken to complete a degree. 
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Figure 2. Graphic depiction of staring dates. 
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Brown (1992) has written up a detailed description of these findings and a copy of her 
report may be obtained from her. Also in the report are several "case descriptions" that give a 
much better feel for the situation than can descriptive statistics. For example: "An individual in 
the sample began as a full-time student in Fall, 1979, and over the next few years attended 5 
semesters full-time, 5 semesters part-time and 3 summers at CCSU. After completing some 
semesters at CCSU, however, the student transferred to a Connecticut community college, 
received an Associate's 
degree, returned to CCSU 
and completed a Bachelor's 
degree, also completing a 
cooperative education work- 
block as part of her/his 
degree program. It is diffi- 
cult to know whether this 
individual should be 'count- 
ed' as a full-time or part-time 
student, and whether he/she 
should be considered as 
"native" to CCSU or as a 
transfer student." It is per- 
fectly clear that this student 
should not be seen as part of 
attrition, although that is pre- 
cisely how he/she was seen. 
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Figure 3. Two models of the college "process " The upper graph represents the 
traditional four- year college, the lower, the modern comprehensive university. 
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Many such patterns of attendance were found. This is not surprising as we exert consider- 
able effort to ensure that students can move in and out of the system easily. It was also clear that 
the commonly applied definitions of retention are not only inappropriate for institutions such as 
CCSU, but that they actually do a great disservice to them. The application of a metric that is 
appropriate to a very different type of college can only lead to erroneous conclusions about these 
public, comprehensive universities that educate so many of our nation's students. A crude 
graphic representation of the different models is shown in Figure 3. 

We concluded our presentation by recommending that other indicators of success be devel- 
oped and adopted at public comprehensive universities; alternatives to the retention and gradua- 
tion-rate statistics better suited to colleges with different missions than ours. The model 
becomes much more complicated, but it must to reflect the complexity of the institutions in 
question. For the same reasons, it is clear that community colleges are ill-served by measures 
that assess their ability to do what they were not designed to do. 

Finally, we argued that the attendance patterns, in all their variety, found at our university, 
reflect a strength — an ability to meet our students' needs — not a shortcoming. A great many of 
our students would be unable to attend and complete their college educations if restricted to the 
traditional attendance pattern: four years, full-time and out. 
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MAKING THE MOST OF THE MISSION REVIEW: 
A TOPICAL CASE STUDY 

Eleanor Fujita 
Director, Institutional Research 
and Mark Oromaner 
Dean, Planning and Institutional Research 
Hudson County Community College, Jersey City, NJ 

The mission statement of a college is visited with some regularity in response to demands 
from accrediting associations and from changes in the internal and external environments. This case 
study outlines the steps taken by one college during its mission review and discusses the role of the 
mission review in the revitalization of that institution. 

The Function of Mission Statements 

The manifest value of college mission statements is open to debate. This is nowhere 
illustrated more clearly than by Warren Martin in his serious but entertaining chapter on the value of 
the mission statement: 

A riddle: What is deep yet lofty, broad yet complex, important yet ignored? Not 
metaphysics, though that's a good guess. Not politics in Argentina, though your 
persistence is admirable. Not a gingerbread house, though now you're getting close. 
The answer? College mission statements. 

The mission statement is the foundation on which the House of Intellect stands. And 
lofty are the utterances that express the importance of our college's mission. Indeed, they 
float like puffy clouds over our solidly positioned edifice. Broad is the applicability assigned 
these statements; so broad that they are thought to cover every contingency. Yet, narrow is 
the gate to understanding them, and few there be that find it. No wonder, then, given the 
mission statement's depth and height, breadth and density, that it is so often ignored. (1985, 
40) 

He goes on, however, to advocate for a mission statement as the opportunity for a college to 
distinguish itself, and as a guiding force for organizational movement, M Despite its flights of rhetoric 
and sweeping generalizations, a good mission statement informs behavior and helps members of the 
community decide when to say no and when to say yes. It is a statement of intention that affects 
practice" (61). 

It has been contended and supported by researchers that a clearly articulated and commonly 
held mission is related to institutional effectiveness and well-being (Smart and Hamm 1993). This 
was found true in two-year as well as four-year colleges by Smart and Hamm. They further found, 
however, that the complexity of the two-year college mission was not necessarily related to 
institutional effectiveness: colleges emphasizing occupational programs, transfer programs and 
continuing education (tripartite mission) and colleges emphasizing only occupational programs 
(singular mission) were both found effective on more indices than those emphasizing occupational and 
transfer programs (dual mission). This would suggest that the complexity of mission and clarity of 
mission are not necessarily incongruent entities. . 
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However, there appears to be disagreement as to the importance of unanimity among 
constituencies on a college's mission. Smart and Hamra note that researchers Chaffee and Ewell 
found agreement to be important for either the "'turnaround* strategies* 1 or "perceptions of 
institutional performance" (490). Others contend as well that "people's perceptions are critical to an 
institution's well-being," and therefore congruency between the desired and perceived image is 
important for marketing, recruiting, and gaining resources (Terkla and Pagano 1993). 

Birnbaum found, however, that agreement on college goals among governing board members, 
presidents, and senior academic administrators and faculty members appeared related to institutional 
type with the least consistency among leaders in universities and community colleges (1988). This 
finding could be construed to suggest that there may be a mediating efiect confounding the 
relationship between goal agreement and institutional functioning. After noting that some would 
argue that goal diversity might reflect confusion about the mission of the college which could hamper 
institutional effectiveness, Birnbaum posited that it could be the contrary: that "pluralistic goals" may 
enhance effectiveness of colleges facing "environmental and programmatic constraints" such as 
universities and community colleges (28). "Goal diversity increases the probability that different 
environmental cues will receive attention and that advocates will exist to support the needs of different 
programs" (29). 

Elsewhere Birnbaum, basing his work on Merton, has explored the manifest and latent 
functions of activities in college governance, particularly the role of academic senates (1989). He 
proposed that senates fulfilled many purposes (as symbol, status provider, garbage can and deep 
freeze, attention cue, personnel screening device, organisational conservator, ritual and pastime, and 
scapegoat), and that these latent functions were valuable to the organization even when manifest 
functions seemed to be lacking. In a similar way, the review of a mission statement, may well serve 
both latent and manifest functions. The following case study of a mission review process illustrates 
this point. 

A Case Study of a Mission Review 

Hudson County Community College, in part because of rapid leadership turnover, a lack of 
permanent facilities, a distinct arrangement for instructional delivery, and comparatively low outcome 
ratings, was under extreme pressure during the Spring and Summer of 1992 to defend its existence. 

In response to a number of forces, e.g. pressure from the state Department of Higher 
Education, recommendations from a locally established Blue Ribbon Panel, and recognition of the fact 
that the existing mission statement had been in place for a decade, the Board of Trustees adopted a 
resolution in September 1992 authorizing the newly appointed president "to implement a process that 
[would] culminate in the submission of a draft revised mission statement to the Board no later than 
April, 1993" (Resolution 1992; Exhibit A). 

Glen Gabert, who took office the same month as the Board of Trustees approved the 
resolution, chose the review of the mission as an opportunity for institutional renewal. In order to 
maximize the; results of this process, a multi-faceted approach was taken. Although the process is not 
complete (i.e. the resulting mission statement is becoming internalized through the planning process 
and we continue to review and share feedback received during the review), over the past eleven 
months the following activities have been conducted. These activities not only resulted in a new 
mission statement in April but continue as a vehicle of institutional reappraisal and revitalization. At 
each step, a continual process of gathering, analyzing, and sharing information and then gathering, 
analyzing, and sharing feedback took place. 
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1. Review of Mission Statements from Other Comprehensive Community Colleges 

The mission of the college had been a limited but evolving one, emphasizing programs 
preparing students for entering occupations. Transfer programs were only recently being phased in, 
and continuing education was limited. Therefore, it was determined that it was important to include 
an educational activity in which the college community could consider how other colleges conceived 
of their mission to their community. 

During the period from October 1992 through January 1993, the Dean of Institutional 
Research and Planning selected a sample of 67 comprehensive community colleges in 22 states and 
gathered mission statements and mission-related documents from these colleges. The resulting 
documents came from 12 of the 19 community colleges in New Jersey (the home state with shared 
licensing authority) and 8 community colleges in New York (nearby and urban with shared 
accrediting affiliation). Beyond this, the criteria for selection included "comprehensive mission, 
diverse population in service area, and national reputation" (National Sample 1993, Introduction; 
Exhibit B). 

The mission statements and related documents were gathered together in two volumes, 
photocopied, and placed in each of the different college locations. 

It was noted in the document that the Monroe Community College (NY) mission statement 

provides an excellent set of questions we may wish to consider in the development of 
the mission statement for the new comprehensive Hudson County Community 
College. These questions are: 

Is the mission statement inclusive? 
Is the mission statement clear? 

Is it succinctly stated in language that all constituents can understand? 
Does it avoid jargon and cloudy rhetoric? 

Does it indicate aspirations that can be accomplished? Is it realistic? 
Can progress be adequately defined and assessed? 

Are the elements those for which the College wishes to be held accountable? 
We may wish to add the following question: 

Does the mission statement address the needs of Hudson County and of its residents? 

College persormel were invited to study the mission statements during the months in which the 
review process took place. Once the first draft statement was circulated, college personnel were 
particularly invited to review these statements in preparation for an all day forum in February. 

The invitation to members of the college community to review these documents put the word 
out that the mission review process was to be an open one: interest was generated. Many persons 
commented on how much they had learned from this review of the varied documents: the educational 
process was initiated. Faculty and staff began to envision new possibilities at Hudson County 
Community College: the revitalization of the college had begun. 
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2. Surveys of College and County Communities 



Early on it was decided that it was important to be as inclusive as possible. A survey of the 
college community and of the county community was planned. Such a survey would serve not only 
to gather information and generate interest in the mission review, but it could also be used to further 
the educational process. 

During November 1992 questionnaires relevant to a mission review were obtained from other 
colleges. A questionnaire obtained from Johnson County Community College (KS) was revised and 
distributed to over 1,100 persons which included all college employees and a sample of students (both 
part-time and full-time), alumni, and community leaders in business or industry, education, 
government, health and human services, and the legal, insurance, and real estate professions. 

The questionnaires had three parts: (1) requests for ratings on the perceived importance of 33 
possible functions, programs, or services the college could offer, (2) requests for demographic 
information, and (3) a request for responses to an open-ended item, "We would appreciate any 
comments you might have about Hudson County Community College". The two versions of the 
questionnaire drawn up for the college and county surveys were identical with the exception of a 
question regarding either college affiliation (e.g. student, faculty) or county occupational affiliation 
(e.g. business, education). (Exhibit C) 

It was possible, from these items, to gather information not only about what respondents 
considered important to include in college's mission, but also to gather information about how the 
college was viewed and whether or not there was agreement among the various constituencies. 
Analysis of the responses resulted in three reports, the first of which reported the contributed 
comments from the college community; the second, the contributed comments from the county 
leaders, and the third, the comparative responses from both groups to the closed items (Mission 
Survey 1993; Exhibit D). 

From the analysis of the responses, ten items were rated "quite" to "very important." These 
emphasized career oriented degree programs and services, transfer oriented degree programs, 
affordability and accessibility, centralized campus, comprehensive curriculum, and job training 
courses. The programs or services considered "least" important (even these items were ranked as 
"important") included providing access to facilities for community groups, sponsoring athletic teams, 
and bilingual instruction. It was found that there was a great deal of agreement between the college 
community and the external community as to the importance of the various programs or services, 
although most were considered more important by members of the college community. 

This feedback from the college and county communities not only provided guidance in the 
drafting of the mission statement, but also served as a caution: two of the programs in which the 
college took great pride were among those with lower rankings. The athletic teams, even without 
facilities to practice, take regional and national championships, and the bilingual programs serve a 
large portion of the college enrollment. The dilemma then is whether or not the college will continue 
these programs; and, since it is quite likely that it will, the challenge is for the college to change 
perceptions about the importance of these programs. 

The resulting reports from these surveys were circulated widely within the college, one has 
been submitted to ERIC, results have been shared at regional symposiums, and further analyses 
continue with results shared. 
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3. Review of College Background Documents 



The College held ar* all-day forum to consider issues related to the mission, and this will be 
discussed later. Background documents were collected together and distributed to the participants so 
that they could fully prepare to take an active part in the forum. This was in addition to the two 
volumes of mission statements from other community colleges described earlier. 

The documents included the following: 

• Draft Revised Mission Statement Resolution of the College's Board of Trustees 
(September 21, 1992). 

• Mission, Goals, and Objectives Statement from Chara cteristics of Excellence in 
Hi g her Education (1982). Commission on Higher Education, Middle States 
Association of Colleges and Schools. 

• A comprehensive Urban Community College; A Vision for the Future (1991). 
Department of Higher Education, State of New Jersey. 

• Philosophy, Mission and Goals Statement from Hudson County Community Colle ge. 
Master Plan, 1986-1991: A Str ategic Vision (1986). 

• Overview of National Sample of 67 Mission Statements from Comprehensive 
Community Colleges (1993). (Described in Step 1) 

• Hudson County Community College Mission Survey Results (1993) (Reports 
Described in Step 2) 

Together these documents provided a broad perspective: they included the college's own 
earlier plan, the vision of the state from which it was licensed, the model from the regional 
accrediting association, versions from other comprehensive community colleges, and the views of the 
community - both within and without the college. 

4. All-Day Mission Forum 

On February 27, 1993, an all-day forum was held at a hotel in the community (Exhibit F). 
Prior to the forum, all who accepted the invitation to attend were given the background documents 
(mentioned in Step 3) and a copy of the first draft of a possible mission statement. 

In all, one-hundred college and county community members attended. College participants 
included students, alumni, faculty, administrators, board of trustee members, and members of the 
support, maintenance, and security staffs. Community members included representatives from 
business, industry, education, health and human services, community agencies, and the legal, real 
estate and insurance professions. The president of the nearby state college joined in the proceedings. 



The president of the college and the Board of Trustee members took an active part in the 
forum. Louis Bender, Professor Emeritus of Higher Education at Florida State University and a 
national expert on community colleges, addressed the group and inspired and challenged participants 
to consider how the college could find and exploit a niche. The group then broke into six smaller 
focus groups to consider specific issues related to the college's future and mission. The groups 
purposefully included a mix of students, faculty, administrators, support staff, maintenance staff, 
security personnel, governing board members and community members. Individual college faculty, 
administrators, staff and students served as facilitators, resource persons, and recorders of each 
group. Following a shared luncheon, a reporter from each group presented a summary of the 
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deliberations to the entire group; these reporters were often community representatives. Following 
the forum, the proceedings from each focus group were sent to all participants. 

This activity generated a great deal of interaction among persons from both the college and 
the county community who otherwise did not have the opportunity to interact. Staff members, who 
might not otherwise have felt that their opinions mattered as to the future of the college, were heard; 
they were enthusiastic in sharing their opinions. And they were impressed that they were able to 
share their opinions in the forum as peers of such leaders as college presidents, 

5. Mission Statement 

A draft mission statement was drawn up based on activities through early February. As 
mentioned earlier it was shared with participants of the forum. It was also distributed to all 
employees for feedback. Following the forum and feedback from the college community, it was 
revised and in March 1993 shared a second time with the college community through publication in 
the employee newsletter, HCCC Network, and with the county community through letters to those 
who had earlier received the mission questionnaire. Again feedback was sought. This resulted in the 
inclusion of a general education goal statement. 

The final mission statement was presented to the Board of Trustees who approved it at that 
body's April 1993 meeting (Exhibit G). 

6. The Aftermath 

Following the adoption of the revised mission statement, much activity has taken place which 
has been a natural outgrowth of the mission review. 

• Short and long term planning has proceeded within the framework of the mission goal 
statements. 

• A paper was prepared on the evolution of the college's mission from a limited mission 
to a comprehensive one; this paper was presented at a staff development seminar in 
September and at a symposium on urban education at Jersey City State College just a 
week ago. 

• A paper was prepared on the community's perceptions of the college (as based on the 
mission survey); this paper was presented at a staff development seminar in October. 

• A paper was prepared on the argument for the provision of developmental education 
as integral to the mission of the college; this paper was also presented at the 
symposium on urban education at Jersey City State College. We are considering 
writing such arguments for each goal of the mission statement. 

• Institutional research articles concerning county demographics in the staff newspaper, 
H£CC Net\york $« have related the analysis to the goals of the mission statement, 

• Articles and announcements are being prepared for submission to higher education, in 
particular community college, and planning and research publications. 

• A one year review of the impact of the new mission statement will serve as the focus 
of a staff development day during the Spring 1994 semester. 

And here and now, the p-ocess is being share with other professionals. The ripple effects 
from our mission review go on. 
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The resulting mission statement is but one outcome of this review process. A message of 
openness has gone out to the college community and the community at large. A vast amount of 
learning about the mission of community colleges, in particular the comprehensive urban community 
college, has taken place. Communication channels among members of the college community and 
between members of the college and county communities has helped to begin the process of dispelling 
old negative images about the college and generate new ideas for the development of programs and 
outreach programs in the community: community representatives have been pleased to observe the 
sense of momentum among college employees, and allege employees have been heartened to realize 
the vast amount of goodwill and support the college has in the county. 

Besides the positive momentum the process has given the college community, some lessons 
have been learned which offer future directions. Feedback from the survey and forum break out 
groups have presented some puzzles for the college which suggest either institutional change or a 
public relations campaign to educate the community. An example of this concerns our bilingual 
program. We are challenged to better explain the necessity of these programs. There seems often to 
be a misconception that somehow our non-English speaking population did not learn the English 
language as they should have in public schools; it is often forgotten that many of them came to 
Hudson County as adults and did not go through the educational system in the United States. This is 
but one example of what we learned from the survey. 

Conclusion 

Hudson County Community College has a new mission statement. The statement documents 
that the college has moved from a limited to a comprehensive mission. The statement symbolizes the 
expansion of the college in enrollment, programs, full-time faculty and facilities. It symbolizes a 
college moving away from its contracted instructional programs to independence. It symbolizes a 
college ready to more fully serve its community. It symbolizes a college on the move. The mission 
review can serve powerfully latent as well as manifest functions for a college. 
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INTRODUCTION 

The origins of the competitive written examination may be traced to 
China's Sui dynasty, in the late sixth and early seventh centuries (Lederman, 
1986). A number of writers have described procedures and problems 
characteristic of this period, which persist today (Anastasi, 1993). These 
include concerns that the form of the test was dictating the content and 
method of instruction (Fairbank, Ref.schauer & Craig, 1973), the influences of 
irrelevant variables on the assessment process (DuBois, 1970), and the 
temptation to cheat (Miyazaki, 1981). 

The use of written examinations in western society may be traced to the 
European university. DuBois (1970) and Ebel (1972) document the evolution of 
the examination process, and the use of examinations in certification, 
evaluation and selection. The movement from an oral to a written process of 
examination, common by the thirteenth century, has a long history (Lunsford, 
1986), and has not been without its skeptics. Despite the dependence of 
contemporary instruction in writing on the Greek tradition, especially 
Aristotle's approach to reasoning (Applebee, 1984), it is interesting to note 
that his mentor, Plato, was an outspoken critic of written discourse, 
regarding writing as both inferior to oral debate and, in fact, harmful to 
one's intellectual development. The debates concerning the relative merits of 
oral and written discourse have continued into the Twentieth Century 
(Lunsford, 1986) , but were somewhat "derailed" by the rise of the "new type" 
or objective examination in the 1920s (Ruch, 1929). The adoption of the 
"objective" format, with its promise of efficiency, reliability and 
scientific rigor, was undoubtedly accelerated by a growing awareness and 
documentation of the difficulties inherent in the evaluation of written 
examinations (Starch and Elliot, 1912) . The dispute evolved from the relative 
advantages of oral versus written discourse, to the relative advantages of 
written versus obj ectively- scored means of evaluation. Another positive 
influence on the growth of objective : terns and tests was the development of 
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commercial standardized tests based in whole or part on the objective format. 
The process is exemplified by the development of the Scholastic Aptitude Test 
(SAT) (Angoff & Dyer, 1971). 

A number of events, documented and discussed by Haney (1984), and 
spanning the period from the mid-1950s to the late 1970s, led to both an 
explosive growth in testing and to critical attacks on the testing 
enterprise. During this period, attention was focused on standardized tests 
of the objective variety used in "high stakes" decision-making. The "war on 
testing" (Lerner , 1980) raged into the 1980s and finally seems to have 
diminished somewhat in intensity. It should be pointed out that the field of 
tests and measurements has had its share of "internal critics", many of them 
leaders in their respective fields (e.g., Dyer, 1977). Despite the huge 
success of standardized testing, and the growing popularity of obj ective 
tests for selection, evaluation and certification, there were indications 
that the isolation and esoteric nature of psychometrics was at least partly 
responsible for much of public hostility toward testing (Hoffman, 1962; 
Anastasi, 1967). In the same vein, Buros (1978) summarized the recent major 
changes in the measurement of achievement to be in the areas of test scoring, 
data analysis and reporting, and not in improved strategies for evaluation 
(p. 1972). 

The 1970s and 1980s also brought the growing realization that there was 
an impending literacy crisis (Purnell, 1982). The apparent decline in 
reading, speaking and writing skills may have been partially created (or, 
perhaps revealed) by higher education's efforts to encourage and enable large 
numbers of "new students" to enter college in the late 1960s and early 1970s. 
The decline (usually in terms of standardized test scores) has resulted in 
both accrediting agencies and institutions emphasizing both instruction and 
improved evaluation in the communication skills of reading, writing and 
speaking. 

The purpose of the present paper is to review the theoretical and 
empirical issues relevant to the evaluation of student writing abilities. The 
emphasis in the paper will be on the use of direct assessment (e.g. writing 
samples, essay tests) as opposed to indirect assessment (e.g. objective tests 
requiring error identification or correction; compliance with rules of 
grammar) . 

The more fundamental reason for the orientation and emphasis of the 
paper is the authors' conviction that most teachers of writing (and many 
critics of objective tests) are correct in their insistence that the 
selection of a correct option or answer on a multiple- choice item involves 
different skills, knowledge and behaviors than the process of planning, 
creating and revising a response to a prompt on an essay test. However, we 
are also convinced that both strategies for evaluating students play a 
critical role in the assessment of writing skill, beyond the often-repeated 
observation that there are modest correlations between the two types of data. 
As White (1986) so succinctly stated, "Different kinds of skills are most 
effectively measured by different kinds of tests" (p. 62). 
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THE DEBATE OVER INDIRECT VERSUS DIRECT MEANS OF ASSESSING WRITING ABILITY 



Proponents of the indirect approach to the assessment of writing 
ability usually cite one or more of the following as advantages associated 
with this strategy: 

1. There are definite time, cost and efficiency advantages in the use of 
objectively-scored tests when the group to be tested is large. 

2. The scoring procedure is highly reliable. 

3. The sampling of a domain of interest can be more thorough. 

4. The evaluation of student responses is impartial. 

5. The procedure is more responsive to pressure for prompt reporting of 
test results. 

6. Test scoring can be accomplished by persons with no knowledge of test 
content . 

Direct approaches are regarded as a more rigorous and intensive 
evaluation of student achievement, especially in the assessment of writing 
skills, and are alleged to have a beneficial effect on student learning. With 
respect to the evaluation of writing skills, the essay test possesses face 
validity, with at least the potential for high fidelity with a variety of 
real-life writing situations and demands (Stiggins, 1982), and appears to be 
an example of work - sample tests (Guiort, 1979). However, as the Standards 
for Educational and Psychological Tests (American Psychological Association, 
1974) points out 

So-called face validity , the mere appearance of validity , is not 
an acceptable basis for interpretive inferences from test scores, 
(p. 26) 

Also, a close consideration of Guion's theoretical development reveals 
significant deficiencies in the typical essay test. 

It is commonly accepted that essay tests typically provide limited 
sampling of student abilities, achievement or knowledge, they are time- 
consuming, expensive and difficult to grade reliably (i.e. , requiring 
trained, knowledgeable raters), prevent prompt feedback to students, and are 
host to a number of other problems. However, the feeling persists that 
students do, in fact, learn from the writing experience. Schumacher and Nash 
(1991) , state , for example , "... that writing is crucial for learning is s, 
well-accepted fact in the education community" (p. 67). The data provided to 
support this statement are complex, confusing and incomplete . Probably 
Applebee's (1984) statement based on a comprehensive review of the research 
literature, that "... the relationship between writing and reasoning has been 
one of the unexamined assumptions in the study of writing instruction" (p. 
590) , is probably a more accurate description of our current state of 
knowledge concerning the relationship between writing and learning. Perhaps 
the central problem with most writing- to-learn research (indeed> most 
research on the evaluation of writing skills) is the atheoretic approach 
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taken by most researchers, an approach that results in their finding that 
different tasks result in different kinds of cognitive operations that, in 
turn, result in different kinds of learning (Durst, 1987; Ackerman & Smith, 
1988; Tierney, Sater, O'Flahavan, & McGinley, 1989). In the absence of 
theoretically-oriented research, studies appear fragmented, inferences are 
severely limited or qualified, and general principles are difficult to 
detect. As summarized by Newell and Winograd (1989), "...there is at present 
only a slender empirical base from which to conceptualize how . . . the writing 
process and what writers take from writing are interrelated" (p. 196). 

One of the most important conclusions that can be drawn from the 
literature concerning the relative advantages and disadvantages of direct and 
indirect assessment is that both approaches have the potential for 
contributing to our knowledge and understanding of student learning, 
retention, and achievement. Objective tests can be designed to systematically 
investigate the acquisition of fundamental skills and competencies (e.g., 
spelling, knowledge of conventions) that provide part of the foundation 
necessary to enable a student to write well (Conlan, 1986). Royer, Cisero 
and Carlo (1993) identify such measurements as useful indices of the extent 
to which learners have acquired skills and knowledge needed to function at 
the most basic stage of skill development. The empirical relationship between 
the mastery of basic skills and the demonstration of the advanced 
competencies (e.g., "good" writing) supposedly dependent upon them needs to 
be studied. 

A common criticism of indirect assessment, as exemplified by the 
multiple-choice format, is that trivia is emphasized. This argument is 
usually based on the lack of face validity for the approach. However, as 
Matalene (1982) has demonstrated, objective tests can be developed to measure 
more complex cognitive skills and abilities. 

Williams (1982) and Applebee (1984) have pointed out that the focus on 
trivial types of knowledge and skills seems to be a persistent problem in the 
direcc approach to assessment as well. Schumacher and Nash (1991) comment 
that knowledge change, in both instruction and research, is often equated 
with knowledge accretion, the simple acquisition of facts. If one of the 
advantages of the direct approach to writing assessment is that students can 
reorganize and reconceptualize information, two requirements seem obvious. 
There should be a deliberate effort on the part of instruction to develop 
these abilities, and the evaluation procedure should provide the opportunity 
for their demonstration and assessment. Perhaps the trend toward "authentic 
assessment" (Wiggins, 1989; Adler, 1993); American College Testing Program, 
1992) and portfolio assessment (Far West Laboratory, 1992) will accelerate 
these developments by revealing the deficiencies in our current approaches to 
assessment (and instruction) . 

"Authentic" approaches to assessment are generally regarded as 
requiring the following properties: 

1. the assessments should utilize real settings, where possible; 

2 . the assessment should require the student to apply skills to the 



resolution of problems which are present both in and outside of the 
classroom; 
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3. the assessment procedure should be validated against relevant and 
significant criteria; 

4. the measurement procedure should address central and significant 
issues, that are potentially of broad scope, complex and ill- defined, 
demanding the application of higher order cognitive skills; 

5 . the evaluative procedure should, in its own right, be a learning 
experience for the student . 

A number of studies have demonstrated substantial correlations between 
indirect and direct measures of writing ability (Breland & Gaynor, 1979; 
Hogan & Mishler, 1980; Coffman, 1966). These studies typically focused on 
questions of "concurrent validity" (i.e., Can direct measures be replaced by 
indirect measures?) without considering the degree to which both measures 
were dependent on some common (third) variable. Huddleston (1954) 
demonstrated that both indirect and direct measures of writing ability may be 
somewhat imperfect indices of verbal ability, a possibility echoed by Breland 
and Gaynor. Bamberg (1982) reported a low correlation between direct and 
indirect measures of writing ability, and a high correlation between the 
indirect measure (objective test scores) and a measure of verbal ability. 
This pattern was interpreted as evidence for the lack of validity of the 
indirect measure and evidence for the validity of the direct approach. 
Obviously, there is some information missing the reliability of the direct 
data and its correlation with the measure of verbal ability. This is an area 
of inquiry that needs to be carefully studied. Finding no correlation between 
direct and indirect measures of writing ability provides a very tenuous basis 
for arguing for the validity of either (e.g. Sabban & Kay, 1987). 

The questions of examiner and examinee preferences for indirect versus 
direct approaches to the assessment of writing ability has received little 
attention. Pumell (1982) found that about 80 percent of the college writing 
instructors who responded to national surveys in 1979 and 1981 reported that 
a direct approach to assessment was an integral and indispensable part of the 
assessment of writing proficiency. Zeidner (1987) investigated student 
perceptions of the relative merits of direct versus indirect measures among 
junior high school students in Israel. The results of two surveys indicated 
a strong preference for multiple-choice tests rather than essays. Despite 
this stated preference, students considered essays as somewhat more 
appropriate than multiple- choice exams for the purpose of reflecting one's 
knowledge in the subject matter tested. Zeidner' s comments concerning 
possible relationships among attitudes , perceptions and preferences of 
students, and test-related behaviors (e.g., preparation, cooperation, 
performance) provide a number of avenues for further study. 

THE USE OF DIRECT ASSESSMENT (THE WRITING SAMPLE) 
AS A MEASURE OF PROGRAM EFFECT 

This portion of the review will focus on the relevant literature and 
research concerning the writing sample as a measurement procedure. The 
writing sample will be regarded as a special case of the more general 
approach to evaluation, the essay test. Although the interest in such a 
sample may be primarily in the writing abilities of students, with content of 
secondary concern, and the interest in the more general essay examination 
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shared between writing skills and mastery of content, many, if not most, of 
the research issues and problems are the same. Therefore, the terms "writing 
sample" and "essay test" will be regarded as virtually synonymous. 

Reliability Issues 

The Sampling of Behavior 
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Concern is often expressed over the adequacy of the sampling of 
behavior from some real or hypothetical population. Limitations result from 
the amount of time required to respond to one or more "prompts" (i.e., 
stimuli) designed to elicit the response from the students . As an area of ^ 
concern, one would classify this as a problem of test design. A moment's ■ 
reflection will reveal that there is not one "population of interest", but ™ 
many. As Odell and Cooper (1980) point out, current discourse theory assumes 
an ability on the part of the student to address diverse audiences, to 
accomplish varied objectives. Variations within task structure, such as 
discourse aim (e.g. , Kinneavy's (1971) expressive, literary, persuasive or 
referential ; Button' s (1978) expressive , literary or transactional ; 
Quellmalz's (1984) exposition, persuasion, narration and description), 
assigned topic, time allowed, the students' perceptions of their audience, as 
well as numerous other factors, taken singly or in various combinations, 
create a huge number of possible "populations of interest". Obviously, 
students cannot be adequately taught to function in all, nor do we possess 
the ability to assess their abilities to function in even a large number of 
them. 



While concern over the adequacy of sampling is valid, there also must 
be an operational narrowing of the scope of instruction and assessment to 
enable both. We must also acknowledge that there are fundamental deficiencies 
in our knowledge of how many variables operate to effect student performance. 
As Hoetker (1982) points out, most instructors in writing feel that mode of 
discourse is somehow related to student performance. Not only are we unsure 
of how this occurs, but we cannot even adequately operationally define "mode 
of discourse". A full rhetorical context (as suggested by Quellmalz (198-4)) 
should assist students but we are unsure of how to supply it. We do know that m 
students often do not follow the instructions given, or even write on topics I 
provided why? (Brossell 6c Ash, 1984, pp. 424-425). 
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Given that there are a large number of "populations of interest", it is 
a decision of both instructional and evaluation design as to the number and 
characteristics of the types of behavior which will be addressed. Assuming 
this design decision has been accomplished, one can turn his/her attention 
toward development of the prompts needed. Crocker (1987) offers a useful 
guide to their development, as well as to the entire process of developing an 
evaluative procedure for assessing writing skills. The technical manual for m 
the Tests of General Educational Development (American Council on Education, I 
1993) also provides valuable guidance for prompt development, as well as ™ 
evaluative procedures. Hoetker and Brossell (1986) offer a model for the 
development of essay examination topics that may be relatively content fair, 
considering the variation in the backgrounds, education and experience of 
students . The approach does not eliminate the problems associated with 
inferences based on small or varied samples of behavior, but may reduce the 
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variability in students' behavior if it is due to a lack of equivalent 
familiarity with essay topics. 



Teachers and evaluators often attempt to make the process of evaluation 
more equitable by allowing students a choice among essay topics. The most 
obvious problem with this approach is a lack of comparability in the data 
resulting from the procedure. If the topics are different, then the criteria 
for performance, the rating procedure, the "population of interest" and other 
relevant factors are different, preventing valid comparisons among students. 
Meyer (1939) demonstrated, too, that students cannot identify those topics 
upon which they will do best. A number of other variables may be involved in 
the selection procedure. As Wiseman and Wrigley (1958), demonstrated, the 
selection of specific topics by English school children was moderated by the 
ability level of the children. Although Wiseman and Wrigley state that 
allowing choice "...probably gives satisfaction to both teachers and pupils" 
(p. 138), the systematic effect of extraneous variables and their impact on 
the utility of student evaluations are usually important (Hilgers , 1982). The 
nature of the issues changes significantly, however, if student writing is 
viewed as a learning experience and not primarily as the basis for 
evaluation. 

A related issue concerns the number of samples of behavior needed to 
achieve an acceptable level of reliability. As Guilford (1954) pointed out, 
the effect should be predictable with the Spearman- Brown formula, assuming 
the assumptions of the formula are satisfied (p. 397). Briefly, the formula 
estimates the effect on a reliability coefficient when the length of a test 
or number of raters is changed by some factor n, which can be a whole number 
or decimal value. Steele (1979) found that an optimal number of writing 
samples may be three, rated by two evaluators. This combination resulted in 
a "peaking" of reliability estimates. Further increases in the number of 
samples and raters resulted in relatively minor gains . Tillman, King and 
Michalczyk (1979) obtained similar results. Fader (1986) and White (1986) 
also recommend at least two independent readings of any sample. 

Agreement of Raters 

There are a number of ways to conceptualize the problem of rater 
reliability. One may be concerned with the agreement of raters with 
themselves (a "test -re test" paradigm) , the agreement of a group of raters 
with each other (a "parallel forms" paradigm) , and the agreement of raters 
(either paradigm) with repeated samples of student behavior from some 
population. Since there are different sources of measurement error in each 
research situation, the reliability estimates will differ. 

Although concern is often voiced about the lack of reliability 
associated with essay-based data, reliability questions refer to data 
contaminated by random error. A close scrutiny of most investigations of the 
quality of essay-based data reveals a concern with systematic sources of 
variation in assigned scores. That is, systematic variations in some factors 
are associated with systematic changes or differences in the estimated 
quality (e.g., scores) of the data associated with that factor. These are 
validity questions since the influence is systematic as opposed to random. 



With respect to reliability questions, the distinction should be drawn 
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between reader consistency - i.e., the between-or-wi thin- -rater agreement in 
grading student products - and the reliability of the data generated by the 
examination itself. It has been repeatedly demonstrated (Ruch, 1929; Traxler 
& Anderson, 1935; Coffman, 1971) that agreement in grading essays is 
typically higher than the reliability of the test itself. In terms of 
obtaining data of acceptable quality (i.e., high reliability), it follows 
that the training of raters to a level of high inter- or intra-rater 
agreement is a necessary but not sufficient condition* Attention must be paid 
to the concerns set forth in Section 1., above, many of which represent 
efforts to reduce sources of error variance in the task set for the student 
(e.g., attempt to specify a full rhetorical context for the student). Other 
concerns represent attempts to increase the proportion of true score variance 
in the resulting data (e.g., increase the number of essays written, or the 
number of scoreable parts) . 

Strategies for Scoring Essays 

There are several contemporary strategies for evaluating essays. Each 
will be briefly described, with its advantages and disadvantages. 

Analytic Scoring 

Analytic scoring is the practice of having each rater evaluate each 
essay using a list of specific scoreable features, with scoring values or 
weights for each. The approach can be traced to Diederich's (1974) evaluative 
scale, which considered the scoring of ideas, punctuation, organization, 
wording, flavor, usa^e, spelling and handwriting. 

This approach may be somewhat time-consuming due to the number of 
evaluative decisions that are required. It also appears to have the promise 
of providing extensive diagnostic information of a highly reliable sort, due 
to the number of scoreable factors for a given essay. It also has the 
potential for focussing raters' attention on specific factors to be 
evaluated, rather than a global, general impression that might be swayed by 
irrelevancies or factors regarded as least important. 

Holistic Scoring 

The early development of holistic scoring occurred almost entirely 
under the auspices of the Educational Testing Service (White, 1985, p. 19). 
While sometimes referred to as "general impression" scoring, the procedure 
has evolved beyond the unguided, uncontrolled evaluative procedure Diederich 
(1974) reported in his earliest experiments. Employing a continuum of 
specificity, precision and variety of scoring criteria employed, analytic and 
holistic scoring are polar opposites. 

Holistic scoring has the following features (White, 1984): 

• The reading and evaluation of essays is strictly controlled and is 
implemented with a trained, highly cohesive group. 

• A scoring guide, that is a direct statement of the characteristics of 
papers at various points on the scoring scale, is developed by those 
responsible for the evaluation, before the raters are assembled. 
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• Papers that are illustrative of the various points on the scale 
("anchor papers") are selected, and are used in the training process to 
illustrate the abstract nature of the criteria in the scoring guide 
(sometimes called a "rubic"). (Daiker & Grogan, 1991). 

• Constant supervision of small groups of raters is provided by "table 
leaders" who do not score papers , but monitor the process to ensure 
consistency . 

• Multiple independent scorings are used to test for consistency and to 
identify problems (e.g., a consistently discrepant rater). 

The reliability (degree of agreement among raters, or of raters with 
themselves) of holistic scoring has been reported to range as high as .70 to 
the .90s (Cooper & Odell, 1977, p. 3; Steele, 1979). However, a number of 
questions concerning the validity of the resulting data have been raised by 
researchers (Nagy, Evans & Robinson, 1988). The results of Myers, McConville 
and Coffman (1966) support Huddleston's (1954) conclusion that a general 
ability construct underlies the ranking of papers by the holistic scoring 
method. Their analysis produced factors that supported a general ability or 
general quality interpretation, that appeared to reflect dimensions other 
than simply content. Myers, et al . also demonstrated that properly trained 
and supervised raters using an holistic approach agreed quite closely on 
overall quality judgements, but that various attributes of a composition 
(e.g. spelling, grammar, organization) appeared to be differentially 
weighted. However , the respective attributes were consistently weighted 
across judges. 

A finding of some importance was the dramatic decline in the 
reliability of judgements at the end of the rating process. The authors 
attribute the decline to fatigue, relief, and a resulting lack of attention 
or vigilance. The degree to which this is effected by the magnitude or 
duration of the evaluative task needs to be investigated. 

In a discussion of Diederich's five factors of ideas, form, creativity, 
mechanics and wording, (Diederich, French & Carlton, 1961) and how these 
factors influence ratings , McColly (1970) points out that little more than 
a generalized halo effect may be reflected in ratings. Page's (1968) analysis 
of the Diederich factors might also be interpreted in this way. The 
intercorrelations among scores reflecting the five factors ranged from .65 to 
.89 (p. 92). However, one may argue that this is precisely the goal if a 
total, holistic score is to have meaning as an index of a unitary construct. 
Certainly, in developing a reliable index, one should strive for positively 
intercorrelated measures , each tapping a slightly different aspect of 
behavior. If the components are independent, the reliability will be low and 
the resulting composite score difficult if not impossible to interpret. 

Steele (1985) cites a number of weaknesses in holistic scoring, 
including the fact that the ratings are not diagnostic . This may occur 
because the ratings are not designed to be diagnostic (only an overall rating 
is assigned) , or because the components of the ratings are too highly inter- 
related to be regarded as having any discriminatory value. Quellmalz (1984) 
is particularly critical of this deficiency in holistic scoring, citing the 
effects of a lack of specificity and informative feedback on student 
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? earning, As she points out, it seems inconsistent that an evaluative 
procedure fails to provide the same level of support for judgements that we. 
expect students to offer in support of the views expressed in their essays 
(p. 67). 

Holistic scoring is undoubtedly efficient in terras of accomplishing the 
task of evaluating essays. Quellmalz cites a study that demonstrates the 
difference between analytic scoring and holistic scoring of a group of essays 
to average about one minute. Wiseman (1956) also points out that holistic 
scoring proceeds at a rate about four times faster than graders following an 
analytic schedule . 

Efficiency, and indeed reliability, are both important concerns, but 
secondary to questions of validity. Charney (1984) has provided a thoughtful 
examination of the question of validity of holistic scores, pointing out that 
the reliability of holistically-scored data may be more perceptual than real. 
Salient but superficial aspects of student writing (e.g. length of the 
sample, handwriting legibility, spelling errors) may inflate the level of 
agreement among raters. Barritt, Stock and Clark (1986) document a number of 
problems leading to both "false consistency", or agreements due to irrelevant 
factors, and to inconsistency, such as highly idiosyncratic evaluative 
criteria and procedures. Holistic scoring has the potential for controlling 
both types or sources of error. 

Huot (1990) summarized the problems often mentioned as characteristic 
of holistic scores as: 

1 . their correlation with irrelevant factors , such as appearance and 
length of essay (Markham, 1976); 

2. a lack of diagnostic information concerning the qualities of student 
writing or instructional effects (Faigley, Cherry, Jollife, & Skinner, 
1985; Odell & Cooper, 1980); 

3. the norm-referenced nature of holistic scores (White, 1985); and 

4. the nature of the training process and the evaluative procedure, that 
distorts the rating process, substituting speed (and superficial 
consistency) for a thoughtful evaluation. 

Primary Trait Scoring 

Primary Trait Scoring was developed for evaluating essays produced in 
response to the 1974 writing assessment of the National Assessment of 
Educational Progress (Lloyd- Jones , 1977). Unlike holistic approaches or the 
analytic approach, Primary Trait Scoring begins, not with an analysis of 
specific papers, but with an analysis of the assignment that prompted the 
writing. The rhetorical context for the writing, the intended audience and 
its nature, and the purpose of the writing are established. The writing task 
is analyzed, in order to determine the strategies important to accomplishing 
the task. 

As a result of this analysis of task performance requirements a set of 
traits, or characteristics, which appear appropriate to the task is 
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formulated. The papers are evaluated against a set of criteria, and not 
simply compared with each other. 

An important consequence of this approach is that different tasks and 
different writing samples must be judged by different criteria. Lloyd- Jones 
explicitly rejects the notion that "writing is writing", and that criteria 
such as quality of ideas, organization, etc. have broad applicability across 
all types of writing (Foley, 1971, p. 803; Odell & Cooper, 1980, p. 40). 
Writing ability means the ability to address diverse audiend :s for diverse 
purposes. Consequently, many and varied samples of different types of writing 
are required for the adequate assessment of an individual - - there can be no 
comprehensive understanding of a person's writing ability if only one sample 
or even a small number of samples is available. 

Primary Trait Scoring appears to have the promise of delivering 
sophisticated diagnostic information concerning student abilities and/or the 
effects of instruction. Royer, Cisero and Carlo (1993) point out in the 
context of cognitively-oriented instructional systems: 

. . .a cognitive task analysis should be conducted before an 
attempt is made to train a cognitive skill, and ic should 
certainly precede any effort to measure the acquisition of a 
cognitive skill" (p. 208) 

As most instructional theorists insist, assessment in a cognitively- 
oriented instructional system must provide diagnostic information and not 
merely the success or failure of a particular instructional event. 
Information must be available to evaluate the reasons for success or failure 
and to enable a plan to overcome deficiencies in learners. It is assumed that 
a typology of errors can be constructed, where certain types of errors are 
indicative of certain weaknesses for which particular types of instructional 
events can be prescribed. 

Although Primary Trait Scoring is relatively demanding of examiners, 
requiring a substantial commitment of time and effort both prior to and 
following the collection of writing samples, it appears to have the greatest 
potential for producing valid, informative data concerning student abilities 
and the effects of instruction. 

Other Approaches 

The preceding three methodologies are the most commonly encountered, 
with holistic scoring by far the most popular. There have been other 
approaches to the evaluation of written work, however, such as Page's (1966, 
1968) attempt to evaluate essays via computers. The orientation of most of 
these other strategies have teen toward research rather than informative 
feedback useful to students ai.d instructors. 

A largely unresolved research question concerns the degree to which the 
different approaches to the evaluation of writing are sensitive to different 
aspects of performance. At the most basic level, one might determine the 
intercorrelation of scores resulting from several scorings of a common set of 
essays. Brossell (1986) cites a study by Winters which compared four essay 
scoring systems including general impression and analytic approaches. It was 
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found that, although the four systems produced reliable data, different 
systems produced different patterns of results with groups which varied by 
level of maturity and ability. 

Given the objectives of instruction and evaluation, one approach may be 
more desirable than another, but it also appears the decision will effect the 
nature of the data obtained from the evaluation, and actions taken on the 
basis of that information. 

Systematic Effects on Essay Grading 

Systematic effects are conceptualized as those factors that can be 
consistently shown to effect the evaluation of essays, by either spuriously 
raising or depressing the rating or score assigned to a specific essay or 
type of essay. These are considered to be problems with the validity of the 
resulting data, since there is some definite probability associated with a 
particular type of error, rather than uncertainty about the sign and 
magnitude of error, as is the case when reliability is a primary concern. The 
following are some of the factors that have been shown consistently to 
effect the evaluations of essays. 

The Effects of Handwriting 

Chase (1968) demonstrated that the quality of handwriting and essay 
grades were correlated by requiring raters to evaluate essays that were 
identical in every respect save penmanship. Hughes, Keeling and Tuck (1983) 
obtained similar results, and also found induced expectancies concerning 
student abilities had a significant effect (elevation and depression) on 
assigned essay grades. Chase (1979, 1990) however, found when teachers were 
given positive expectancies concerning student abilities, those essays 
written in a poor hand were rated higher than those written in a good hand. 
These results do not necessarily contradict those of Hughes, Keeling and 
Tuck, but may simply reflect a generalized "halo effect", or the tendency to 
give ostensibly good students the benefit of the doubt whet, unable to easily 
read and understand their work. Another possibility is rater over- 
compensation for the negative affect created by the task difficulty 
associated with reading papers written in a poor hand. 

A number of writers (e.g. McColly , 1970) have posited handwriting 
quality as one of the principal extraneous factors responsible for the 
reported high reliabilities of hoi is tically- scored essays . The obvious 
solution to this problem is to remove the source ol variance, by either re- 
typing essays prior to evaluation or requiring them to be typed when 
submitted. There are obvious problems with either approach. The problem may 
disappear with the growing availability of word processors and printers, or 
may evolve to investigations concerning the effects of type face, pitch, 
ribbon quality and other variables that impact the reader of typed material. 

OiTder or Context Effects 

Context effects refer to the influence of preceding essays, of greater 
or lesser quality, on the evaluation of a given essay in a series being 
evaluated. Coffman and Kufman (1968) and Klein and Hart (1968) failed to find 
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significant context effects. However, a number of studies, summarized in 
Hughes and Keeling (1984), have demonstrated the occurrence of context 
effects in the grading of essays. Typically, if an essay is preceded by an 
inferior example, it is rated higher than if preceded by a superior example. 
These results are quite consistent (Daly & Dicks on -Markham, 1982; Hales & 
Tokar, 1975), even when raters were specifically warned of the existence, 
nature and operation of the effect. (Hughes , Keeling & Tuck 1980 , 1986) . 
Hughes and Keeling (1984) attempted to prevent the operation of context 
effects by providing graders with model essays to which they could refer 
during the grading process, but even this strategy failed to prevent context 
effects . 

In large scale assessments, the only possible means to reduce context 
effects may be multiple readings , with papers randomly reordered between 
readings and the final score an average or simple sum of ratings. Context 
effects would likely persist within ratings, but could be expected to 
diminish across rating events in the summing or averaging process. Basically, 
an attempt would be made to change a systematic effect into a random effect. 

Attention to some of the basic notions concerning the administration 
and scoring of essay tests (e.g., blind scoring to reduce "halo effects"), as 
recommended in standard references (e.g., Gronlund, 1985; Wiersma & Jurs , 
1985), and summarized by Coker, Kolstad and Sosa (1988) may be utilized to 
alleviate some of the problems identified by Hughes and Keeling. 

Topic Familiarity 

Another concern often voiced (Hilgers , 1982) is students ' relative 
familiarity with the topic(s) involved in the essay prompt. In addition, as 
Hoetker (1982) points out, the "mode" of writing called for by any essay 
topic is M . . .precisely that mode that any particular student interprets it as 
calling for." (p, 379). To further complicate the situation, any particular 
student may not be able to or may not care to write in the mode he or she 
interprets a topic to be calling for, or on the topic involved in the prompt. 
In a closely-related study, Brossell and Ash (1984) found that systematic 
variations in the wording of topics and the charge to students failed to 
effect writers or the holistic ratings assigned their essays. 

However, within any group of students, variation in topic familiarity 
may interact with other aspects of the evaluative situation to produce 
complex interactions with factors such as student characteristics (Smith, et 
al . 1985). Steps should be taken to minimize topic familiarity as an 
extraneous source of variance in performance. Hoetker and Brossell (1986, 
1989) describe a method for developing "content fair" essay topics for the 
evaluation of college-age students, and report research evidence that the 
approach minimizes the problem of content familiarity. 

Their approach employs a model that stipulates the general task for the 
student. The student supplies the specific subject to be the focus of the 
essay, as well as any specific definitions involved, such as "significant 
effect" . An example of a topic form might be 

A book that many students read that may affect them positively 

A student would supply the name of a specific book. As Hoetker and 
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Brossell emphasize, the procedure is intended to produce content -fair topics 
which are not content- free . The test developer is still responsible for 
exercising judgement concerning the adequacy of "frames" or models for 
topics , as well as the resulting topics , considering the nature of the 
students involved. 

Length of Response 

Bracht and Hopkins (1968), Klein and Hart (1968), Garber (1967), Grobe 
(1981) and Lonka and Hikkonen (1989) found that the length of response 
correlated substantially with the rating assigned. Hov/evur, Bracht and 
Hopkins (1968) found no relationship between length of writing sample and an 
independent measure, a test covering the same material. This result suggests 
that those who wrote more did not necessarily know more. 

Lonka and Mikkonen hypothesize that the length of a writing sample may 
be a function of the extent of elaboration in a student's response rather 
than a function of the simple recall and reproduction of discrete factual 
information. There is support for this notion in the work begun by Page 
(1966) concerning the structural analysis of essays. Further support, may be 
found in Malgady and Barcher's (1977) investigation of creativity judgments. 
These researchers found that the number and the novelty of sentences were 
both directly related to creativity ratings of a set of essays. 

If holistic ratings (or any other type of rating) are significantly 
effected by perceived creativity, then the relationship between length, as 
an index for creativity or quality, and positive ratings is understandable. 
This interpretation is consistent with Grobe 's (1981) study of compositions 
produced by fifth, eight and eleventh grade students. Grobe found that longer 
compositions that were free of mechanical errors (especially spelling 
mistakes;) were assigned higher grades. A secondary analysis indicated that 
the total number of different words in a composition, potentially reflecting 
fluency, verbal ability or creativity, rather than simple length, predicted 
essay ratings. 

The Infliience of Content 

A number of studies have indicated uhat in evaluative situations which 
are designed to focus raters' attention on content and organization, 
extraneous factors , such as mechanical features of student writing, are 
relatively unimportant sources of variance. 

However, in a relatively uncontrolled evaluation of ever 200 essays 
rated by ten evaluators, Remondino (1959) obtained four factors in an 
analysis of the ratings: 

1. graphic representation (the physical appearance of the paper); 

2. language usage (e.g. spelling, grammar); 

3. content and organization; and 

4. originality, creativity and imagination. 

The four factors, which are listed in descending order in terms of 
variance accounted for, are ranked contrary to the order most instructors or 
evaluators would assign on the basis of relevance or importance. 
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Remondino's first two factors are typical of the type described by 
Charney (1984) and Freedman (1979) , which may act to spuriously inflate the 
reliability of holistic ratings. 

Diederich, French and Carlton (1961), in a more controlled study, found 
ideas, form, flavor, mechanics and wording (in that order) were the primary 
influences on raters ' judgements . Similarly, Freedman (1979 ) , Boodoo and 
Garlinghouse (1983) and Breland and Jones (1984) found content and 
organization to be primary influences. Rafoth and Rubin (1984) concluded that 
mechanics appeared to exert a stronger influence on judges' ratings than did 
content. However, the judges in their study appeared unable to distinguish 
clearly between content and mechanics, and the authors hypothesize that a 
"diffuse reaction" to one factor may have had an influence on the effects of 
the second. Chase (1986) reported complex interactions of extraneous 
variables in the rating situation, potentially indicating that content may 
play a minor role in determining essay grades in some situations (Rafoth & 
Rubin, 1984) . "Blind scoring" will eliminate many of the potential sources of 
bias (e.g. gender, ethnicity, family name) that result in complex 
interactions of the type reported by Chase. A fruitful approach to exploring 
the evaluative processes employed by raters may be protocol analysis or a 
"think- aloud" approach (Vaughan, 1987) . 

Other Factors Influencing Ratings 

Studies have repeatedly demonstrated that readers of essays are unable 
to rate responses independently of errors in spelling, punctuation and 
grammar (Scannell & Marshall, 1966; Marshall, 1967; Marshall & Powers, 1969). 
As pointed out earlier, these skills or knowledge of convention may be 
regarded by raters as basic to the act of composition and raUers may over- 
react to these errors in a writing sample. The presence of these errors may 
also create systematic difficulties in reading and evaluating the essay, 
leading to a systematic relationship between reading ease and assigned scores 
(Chase, 1983). 

Brown (1986) describes a study done at the University of Houston which 
demonstrated systematic differences in the writing patterns of black and 
white students . Black students used over four times as many quotes and 
misquotes, biblical references, metaphors, proverbs, maxims, cliches and 
other distinct features as whites. 

If dialectic features influence holistic scores, by effecting readers' 
general impressions of quality, then a systematic effect on student scores 
will result. The nature of the raters charged with the evaluation of written 
work is an important part of the entire process. Although rater variation may 
be minimized through selection, training and close supervision, the 
possibility exists that certain rater characteristics may interact with the 
rating process to have a systematic influence on the resulting ratings . 

Townsend, Kek and Tuck (1988), for example, investigated the influence 
of raters' mood on assigned ratings. Despite evidence that the ratings were 
completed reliably, and that the variable of mood was manipulated, there was 
no evidence of a systematic effect on ratings. 
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Trueman and Berrisford (1981) 



howe ve r , found a 




significant association between personality (as operationalized by a 
questionnaire) and marks awarded an essay. This may illustrate the relative 
importance of "state" versus "trait" variables in the evaluative process. 



The development of a written plan seems to assume a linear process in 
writing. Perhaps a recursive model, as proposed by Flower and Hayes (1981), 
is more consistent with the demands of the writing task, with the development, 
of the plan and the actual writing two marginally- related behaviors. As 
Bartholomae (1980) demonstrated, students will often spontaneously correct: 
errors in their written prose as they read it, unaware of the errors and 
unable to detect them. 
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Summary 

The conclusions one reaches after a consideration of the theoretical 
and empirical work on the assessment of writing ability depend to a large 
extent upon one ' s initial position and sentiments concerning the central 
issues. Certainly, research would be more fruitful if cast within a theory of 
composition. Short of a fully- developed theory to guide studies in the area, 
perhaps the most incisive approach to evaluation is to employ Lloyd- Jones' 
Primary Trait Scoring procedure. Holistic scoring seems to be a highly 
questionable process, despite producing apparently reliable data. Analytic 
scoring seems to require excessive time to employ, although the procedure 
appears to be capable of providing diagnostic information. 
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Some Factors Influencing Performance 

Torrance, Thomas and Robinson (1991) found that 86 percent of their I 
research participants (secondary school students) believed that the best * 
method for producing a written essay was to first develop a written plan. 
However, data indicated no significant association between the grade awarded 
and plan writing, plan type or the student's reason for planning. Soltis and 
Walberg (1989) found that those teen-aged students who reported engaging in 
prior planning of essays or in substantial revision of written work had lower 
levels of achievement than those who did not. The nature of the prior 
planning or revision, and the degree to with which it was beneficial to the 
final product were not determined. 
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As Soltis and Walberg point out, in contrast to original composition, 
revision receives little time or attention in the writing curriculum or in 
the process of evaluation. Students often do not revise work, but discard it: 
and write an entirely new piece, in need of as much revision as the original. 
Students may perceive the task of revision as an attempt to adhere to _ 
"convention" (e.g., spelling, sentence structure) rather than focusing on a I 
more effective means of expressing themselves. An important finding reported 
by Soltis and Walberg (1989), supporting the results of Kare.gianes, 
Pascarella and Pflaum (1980), was that peer feedback and discussion of 
written work was associated with enhanced levels of achievement. Given the 
repeated accusation in the literature that teacher comments are scanty, 
uninf ormative and rarely considered in re-evaluation of student work, peer 
discussion and feedback for the purposes of revision as an evaluative 
strategy may be an efficient and powerful means of both engaging students 
and improving the quality of their writing. 
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If one accepts the idea that indirect measures of relevant basic skills 
(e.g. spelling, mechanics, error identification and correction) can predict 
the performance of higher -order behaviors best assessed through direct 
measures, then the value of both indirect and direct assessment of writing 
ability is indisputable. The correlation between direct and indirect measures 
and the resultant arguments concerning the primacy of either assume a more 
appropriate perspective . 

Writers who focus on the problems of evaluating essays have often noted 
that individual raters have to make a serious effort to reconcile philosophic 
differences in order to reach consensus. As Daiker and Grogan (1991) point 
out , reaching consensus always means the suppression or exclusion of some 
point of view. Without a common perspective concerning the basis for 
evaluation, the likelihood exists that individual raters are attending to 
different aspects of student performance and weighing the information in a 
highly idiosyncratic fashion. Such a process will never result in acceptable 
inter- rater agreement concerning the evaluation of student writing. This will 
be especially true if raters' attention is directed to specific features of 
written work, rather than a global impression which may be effected by 
extraneous factors. Some restriction in the gene rali z ability of the resulting 
data can be expected, because of the limits inherent in, or placed upon, the 
work of a focused, consistent group of raters, but limited valid inferences 
are preferred over invalid conclusions, or worse, none at all due to the 
unreliability of data. 

A number of research questions, especially those considering the 
cognitive process of composition, are deserving of intensive attention. As 
Facione (1990) pointed out in a consideration of the advantages and 
disadvantages of essay and objective approaches to assessment: 

"...the essay omits claims considered and judged irrelevant, 
arguments evaluated as not of sufficient significance to the 
issues at hand to warrant mention, evidence queried but not used 
in the final form of the essay f alternatives con jectured but 
ultimately abandoned, and conclusions drawn but ultimately 
reconsidered and disregarded ." (p. 7) 



These deficiencies in our attempts at product evaluation are 
representative of the types of research questions that bear directly on the 
supposed superiority of direct methods of assessment, especially their 
effects on learning. Another approach to evaluation, more pragmatic, is 
portfolio assessment. Certainly, a reconceptualization of how writing samples 
are produced, revised and evaluated is needed for their inclusion in student 
portfolios, (Far West Laboratory, 1992), but this appears to be a fruitful 
approach. The few studies that have appeared showing strong, positive effects 
of peer evaluation, guidance and feedback should be continued. As mentioned 
by Adler (1993), the trend toward "authentic assessment" and portfolio-based 
evaluation will require a re -thinking of the classic conceptualizations of 
reliability and validity. 
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The Admitted Student Questionnaire (ASQ) program was developed by the 
College Board in 1988 to offer colleges the opportunity to conduct college 
choice research on their admitted freshmen . The ASQ asks students to 
consider 20 college characteristics, to indicate how important each was in 
their decision, and how the client college (the college sending them the 
questionnaire) rated in comparison to the set of other colleges they 
seriously considered. The survey also covers sources of information about 
the college, perceived images of it, financial aid and cost, and demographic 
information. In addition, the ASQ asks students to list up to six other 
colleges to which they applied, providing the client college with vaJuable 
information on application and admission overlap. 

While the ASQ thus offers the client colleges a good view of the broad 
context within which their admitted students are making a college choice, 
many colleges wished to take the analyses a step further in order to under- 
stand the processes by which students choose between them and specif ic other 
colleges. In 1992 the College Board added to the ASQ program the Admitted 
Student Questionnaire Plus (Plus) to try to meet this need. The Plus covers 
the same topics, but instead of asking for comparative ratings (from Worst to 
Best) of the client college, it asks that students rate the client college 
and up to two other colleges, specified by the respondent on an absolute 
scale (from Poor/Fair to Excellent). 

Because it is always clear on the Plus which colleges the student is 
rating, this instrument offers a unique opportunity to examine the 
consistency of student ratings across questionnaires, that is, when the 
student completes a questionnaire from two or more client colleges and rates 
the same college or colleges on each survey. This paper describes the 
methods used to identify the students completing more than one questionnaire, 
to build the data file, to choose appropriate comparisons, and to analyze the 
data. 



Description of the ASQ Plus instrument 

The ASQ Plus is comprised of seven sections: respondents are asked to 
1) mark how important each of 16 college characteristics was in their college 
choice; 2) provide information on the colleges they applied to, including the 
names of their first, second and third choice colleges; 3) rate the client 
college (CC; called "Our college" on the questionnaire) and up to two other 
colleges (which they name as College A and College B) on the same 16 charac- 
teristics; 4) circle, from a list of 20, the words or phrases which they 
believe to be widely-held images of CC and colleges A and B; 5) rate CC and 
colleges A and B on the quality of information about the college provided by 
12 different sources; 6) indicate their financial aid status at and rate the 
cost of attending CC and colleges A and B; and 7) provide descriptive 
information about themselves (gender, test scores, income, etc.). 

For the third, fourth, fifth, and s^xth sections, described above, the 
respondents are asked to provide three ratings — of CC and colleges A and B. 
They are also asked to write in, for each section, the name of the college 
being rated therein, and are specifically instructed to " Please continue to 
rate the same colleges as A and B 4 hroughout the questionnaire." Almost all 
students do follow that last instruction, but there are a few who switch 
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colleges A and B for one or more sections, or who leave one or more sections 
blank for either or both colleges. A handful of students also rate 
completely different colleges in one or more sections. Because the college 
being rated in each section is specifically keyed along with the ratings, it 
is possible to use all the data, even when the student has failed to follow 
directions. 



Methodology 

The project encompasses four major steps: 1) identifying the students 
who rated the same college(s) on more than one questionnaire; 2) creating a 
single data file from the multiple questionnaires; 3) setting up comparisons 
that were appropriate to the nature and structure of the data; and 4) 
identifying statistical analyses that were also appropriate, given the 
restricted range of the data. This paper will focus on the first three 
steps: the small number of matched cases available in this preliminary study 
(q.v. ) makes detailed statistical analysis tantalizing but inconclusive. 

1 • Identifying the sample . 

Colleges participating in the ASQ Plus (or the ASQ) have the option of 
placing some kind of an identification code on their surveys . This allows 
them to match ASQ data to institutional data for additional analyses. 
Questionnaires can only be matched across colleges if the same IDs are used 
by each college (or if the colleges are able to provide conversions of their 
unique IDs to some common form). Social Security Number (SSN) seemed to 
offer the best possibilities as common IDs, and questionnaires from the 1992 
Plus studies were searched for nine-digit numbers that could be assumed to be 
SSNs. The results are as follows: 

— Of the 32 participating colleges, covering approximately 62 , 000 
respondents, 54 colleges ( approximately 43 , 000 students ) used 
some kind of ID number on their questionnaire. 

— Of those, 14 colleges (comprising 15,000 respondents) used a 
nine-digit number , that was assumed to be an SSN after 
suspicious-looking numbers (such as those beginning with several 
0's) were eliminated. 

— Of those 15,000 nine-digit numbers, 393 were duplicates: 195 
numbers appeared twice, and one appeared three times. In the 
latter case, only two questionnaires were used in the analysis, 
those being the most complete data. 

It was actually not surprising that the number of duplicates was small 
in this preliminary study , because the 14 colleges represented the entire 
spectrum of American higher education, including both public and private 
colleges, large and small, highly selective and less selective, two-year and 
four-year, liberal arts and business. 1 

2 . Creating the data fil e. 

The methods used to create the data file were fairly primitive and 
cumbersome' The original Plus data file of over 62,000 cases was reduced to 
15,000 by .electing only the 14 colleges using a nine-digit ID (using SPSS- 
PC*; a college is identified in the data file by the four-digit code a£. _gned 
to it by the College Board). A frequency count of the ID field revealed the 
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l An independent estimation of the likelihood of duplicates can be made from 
an examination of the colleges admitted to by the respondents from each 
participating college, in order to determine whether other Plus users appeared 
among the colleges listed most frequent ly . 
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196 ID numbers that had a frequency greater than 1. (This was further 
complicated by the fact that one of the colleges had added one letter to the 
end of the ID field, necessitating the conversion of this field from 
alphanumeric to numeric before the IDs could be matched. ) 

An SPSS SELECT IF command extracted the desired 196 IDs, then one file 
of 196 cases was created from the 392 questionnaire records by treating the 
second questionnaire as the second record for each case. (The designation of 
"first" and "second" referred strictly to the order in which they appeared in 
the file, and had no reference at all to the identity of the college mailing 
either survey. ) Elimination of duplicates (where there were two 
questionnaires for a single student from the same college 2 ) and 
questionnaires that had no colleges rated in common, further reduced the 
matched sample to 135. 3 

The variables to be compared include: 

— importance and quality ratings of 13 college characteristics' 5 
a marked/not marked flag for each of 14 college images 

— quality ratings of 12 sources of information about the college 

— a 1-8 rating of the cost of attending the college 

It is also possible to examine whether the respondent listed the same three 
colleges on both questionnaires, in the same order, as his/her top three 
choices, but that discussion will not be included in this paper. 

3 . Establishing comparison sets . 

Identifying the comparisons that could be made from this data set was 
the most difficult part of the study, because of the different combinations 
of data available. For each case (each student) there were six possible sets 
of ratings: on each of the two questionnaires there were ratings of "our 
college" (the CC that had mailed the survey) and two other colleges named by 
the student, referred to on the questionnaire as Colleges A and B. The 
college named and rated as College A was frequently the college to be 
attended by a student who would not matriculate at CC, or the second choice 
of a matriculating student. The basic task was to derive a streamlined set 
of matched ratings from all the possible combinations. 

Since the college that mailed the first questionnaire (CCl) was always 
different from the college mailing the second questionnaire (CC2), the two 
records never contained data for the same colleges located in exactly the 
same fields. They could, however, contain ratings of the same colleges 
located in different fields. Some of the possible combinations are described 
below. 

a) CCl was coded as College A or College B on the second 
questionnaire (CollA2 or CollB2, respectively). For cases 
falling into this category, the ratings of CCl would be compared 



Participating colleges are presumed to have cleaned up their data and 
eliminated duplicate questionnaires before transmitting everything for 
processing . Applied Educational Research does not search for and eliminate 
duplicate records unless requested to do so by the participant. Comparison of 
duplicate questionnaires from the same college would make an interesting study 
in itself. 

because of the large number of variables to be examined, and the convoluted 
nature of the process, SPSS for Windows was used for all subsequent procedures. 

4 The first 13 characteristics shown on the questionnaire, and the first 14 
images listed, are common to all Plus questionnaires. Characteristics 14-16 and 
the 15th-19th images are chosen by each college and are disregarded here. 
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to a new set of ratings consisting of the ratings of either 
CollA2 or CollB2, which represent the same college rated as CC1 . 
The original layout would appear as follows, (This and 
subsequent examples use fictitious college codes.) 

Case 1: CC1: 2222 CollAl: xxxx CollBl: xxxx 

CC2 : xxxx CollA2 : 2222 CollB2 : xxxx 

Case 2: CC1: 2222 CollAl: xxxx CollBl: xxxx 

CC2 : xxxx CollA2 : xxxx CollB2 : 2222 

For this category, the identity of the colleges in the other 
positions was irrelevant. The question of interest here was, 
"With which ratings on the second questionnaire should CC1 be 
compared? " 

b) CC2 was coded as College A or B on the first questionnaire 
(CollAl or CollBl) . This situation is similar to the first in 
that the CC on one of the questionnaires is rated as College A or 
B on the other, but the two did not always occur together. For 
example: 

Case 1: CC1: 2222 CollAl: 3333 CollBl: xxxx 

CC2: 3333 CollA2: xxxx CollB2 : 2222 

Case 2: CC1: xxxx CollAl: xxxx CollBl: 3333 

CC2 : 3333 CollA2 : xxxx CollB2 : xxxx 

These two situations represented the same type of comparison (CC 
on one questionnaire was rated as College A or B on the other) . 
Because they could both exist for the same respondent, however, 
the ratings described in (b) were used to supplement, but not 
substitute for, those used in (a). 

c) Colleges A and/or B were rated on both questionnaires ( indepen- 



dent 


of 


which colleges 


were rated 


as CCs) . 


For example: 




Case 


1: 


CC1: 


xxxx 


CollAl: 


2222 


CollBl: 


3333 






CC2 : 


xxxx 


CollA2: 


2222 


CollB2 : 


3333 


Case 


2: 


CC1 : 


xxxx 


CollAl: 


2222 


CollBl: 


3333 






CC2 : 


xxxx 


CollA2: 


3333 


CollB2 : 


2222 


Case 


3: 


CC1: 


xxxx 


CollAl: 


xxxx 


CollBl : 


2222 






CC2: 


xxxx 


CollA2: 


2222 


CollB2 : 


xxxx 



For 35 of the cases (26%) all three colleges rated on one questionnaire 
were rated on the other. For another 41 (30%), one of the CCs was rated 
along with a different College A or B. The largest group (47, or 35%) rated 
Colleges A and/or B on both questionnaires, but neither of the CCs, The 
remaining 12 cases (9%) had one or both CCs in common, but no other colleges. 
The following comparison also illustrates the amount of overlap among the 
categories and the difficulties involved in matching the ratings correctly: 



a) 


CC1 a Coll2A 


37 


(27%) 


b) 


CC1 = Coll2B 


28 


(21%) 


c) 


CC2 = ColllA 


48 


(36%) 


d) 


CC2 = ColllB 


22 


(16%) 


e) 


ColllA = Coll2B 


1 


( 1%) 


f) 


ColllB = Coll2A 


3 


( 2%) 


g) 


ColllA = Coll2A 


4 


( 3%) 


h) 


ColllB = Coll2B 


39 


(29%) 



Groups (e), (f), and (g) were small because most of the matches involving 
College A occurred in groups (a) and (c). 



EMC 



125 152 



For each section of the questionnaire eight sets of ratings were 
created. (For the importance ratings, which were independent of the colleges 
rated, only two sets were created — one for each questionnaire): (1) ratings 
of the college listed as CC on the first questionnaire; (2) ratings of the 
same college when it was either College A or College B on the second 
questionnaire; (3) ratings of the college listed as CC on the second 
questionnaire; (4) ratings of that college whe:'i it was either College A or 
College B on the first questionnaire; (5) ratings of a college that appeared 
as College A on the first questionnaire and as College A or B on the second 
survey; (6) ratings of the same college from the second questionnaire; (7) 
ratings of the college that appeared as College B on the first questionnaire; 
and (8) ratings of that college from the second questionnaire. 

Ratings of types (1) and (2) apply to comparison categories (a) and 
(b) , described above* Similarly, ratings (3) and (4) apply to (c) and (d) . 
Ratings (5) and (6) represent categories (e) and (g) , while ratings (7) and 
(8) refer to (f) and (h). 

Because most respondents had more than one set of ratings, another data 
file was created which contained one record for each of the four possible 
sets. Specifically, the first of the four records contained ratings of a 
college appearing as both CC1 and Coll2A/Coll2B; the second record contained 
the ratings of any college appearing as both CC2 and ColllA/Colllb; the third 
held ratings of a college appearing as ColllA and Coll2A/Coll2B; and the 
fourth dealt with a college appearing as ColllB and Coll2A/Coll2B. Any given 
respondent would have data for at most three of these sets, since only three 
colleges were rated per questionnaire . 



Results 

Once the myriad data transformations had been completed , it was 
possible to conduct some preliminary analyses. Obviously, the small number 
of duplicate cases available (370 records used out of a total of over 62,000) 
would make it ludicrous to draw any conclusions from the analyses. The 
results described here, however, will help to frame the questions asked in a 
more complete version of the study to be conducted early in 1994. 

Given that the overall question is whether — and to what extent — a 
student's ratings of colleges on the ASQ Plus are consistent from one 
questionnaire to another , more specific research questions deal with the 
individual sections of the questionnaire and with the type of comparison 
being made: 

— I s the importance ascribed to the college characteristics the 
same on both questionnaires? 5 

— Are the ratings of the quality of the characteristics and the 
quality of information provided at a given college consistent 
across questionnaires? 

Do the ratings of a given college tend to be higher if that 
college is rated as CC? In other words, is there any tendency 
for respondents to inflate ratings of the college asking the 
questions? 

Do the respondents mark the same images of the colleges on both 
questionnaires? 

Is the cost of attending the college rated the same both times? 



ERLC 



5 It is assumed here that the importance of the characteristics in the 
college choice decision is independent of which college is asking the question, 
but it is possible to construct circumstances under which that would not be true. 
If, for example, a student were admitted to 10 colleges, Part of the Country 
might be important in eliminating five of the choices. Cost or Academic Repu- 
tation might then become more important in deciding between the remaining five. 
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Table 1 compares the two importance ratings of the 13 college charac- 
teristics, which are listed in the order given on the questionnaire. Only 
two of the differences are statistically significant, but the rating's of all 
characteristics are significantly correlated. The highest correlation is 
between the two importance ratings of Cost of Attendance. Note that all 13 
characteristics are considered at least somewhat important on the average 
(mean rating > 2.0). 

Table 1 

Comparison of Importance Ratings 



Characteristic 


Meanl 


Mean2 


Difference Corr 




Academic reputation 


2.79 


2.85 


-.0579 * 


.710 


* * 


Availability of majors 


2.69 


2.73 


-.0373 


.701 


* * 


Special academic programs 


2.26 


2.21 


.0522 


.554 


* * 


Personal attention 


2.74 


2.75 


-.0075 


.513 


* * 


Academic facilities 


2.61 


2.67 


-.0602 


.485 


* * 


Recreational facilities 


2.41 


2.49 


-.0746 


.542 


* * 


Quality of campus housing 


2.51 


2.50 


.0149 


.467 


* * 


Surroundings 


2. 53 


2.46 


.0677 


.363 


* * 


Campus attractiveness 


2.48 


2.43 


.0522 


.553 


* * 


Cost of attendance 


2.35 


2.24 


.1119 * 


.765 


* * 


Quality of social life 


2.48 


2.49 


-.0075 


.568 


★ * 


Off-campus opportunities 


2 ,23 


2.21 


.0150 


.568 


★ * 


Extracurricular activities 


2.61 


2.50 


.1119 * 


.433 


★ * 


Scale: 1 = Not Important; 2 


- Somewhat 


Important ; 


3 = Very 


Important 





* p < .05 ** p < .01 



Table 2 displays the mean quality ratings for all comparisons (i..e., 
regardless of whether the college being rated was the CC) . Once again, all 
the paired ratings are correlated to a statistically significant degree, and 
now half the differences (seven out of 13) are also significant. Little 
information can be gained from the fact that all but one of the differences 
are negative, since some of the comparisons involve ratings of colleges that 
were not CC for either questionnaire. 



Table 2 



Comparison 


of Quality Ratings, all 


Colleges 








Characteristic 


Meanl 


Mean2 


Difference 


Corr 




Academic reputation 


3.45 


3.48 


-.0284 




.453 


* * 


Availability of majors 


3.40 


3.43 


-.0287 




.561 


* * 


Special academic programs 


3.09 


3.51 


-.4130 


* * 


.269 


★ * 


Personal attention 


3.20 


3.32 


-.1145 




.499 


★ * 


Academic facilities 


3.28 


3. 54 


-.2604 


* * 


.191 


★ 


Recreational facilities 


3.15 


3.42 


-.2761 


* * 


.403 


* * 


Quality of campus housing 


2.92 


3. 16 


-.2468 


* * 


.408 


* * 


Surroundings 


2.91 


2.90 


.0121 




.602 


* * 


Campus attractiveness 


3.41 


3.53 


-.1205 




.405 


* * 


Cost of attendance 


2.48 


2.59 


-.1090 




.492 


* * 


Quality of social life 


2.91 


3.26 


-.4490 


* * 


.382 


* * 


Of f -campus opportunities 


2.98 


3.21 


-.2267 


* 


.341 


* * 


Extracurricular activities 


3.26 


3.46 


-.2062 


* 


.330 


* * 


Scale: 1 - Poor/Fair, 2 = 


Good, 3 = 


Very Good, 4 - 


Excellent 









* p < .05 ** p < .01 



Table 3 analyzes only the ratings of colleges that were the CC for one 
of the questionnaires. In this table, the column labeled Meanl shows the 
mean ratings of the college when it was CC1 (that is, when it was the college 
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asking the question), and the Mean2 column shows the ratings of the college 
when it was rated as College A or College B. Far from being inflated, the CC 
ratings are actually lower than the A/B ratings for 10 of the 13 
characteristics, although only five of the differences are significant. 

Table 3 



Comparison 


of Quality 


Ratings, 


CC Only 






Characteristic 


Meanl 


Mean2 


Difference 


Corr 






3 .46 


3. 


45 


. 0152 


* 565 


* * 


Availability of majors 


3.38 


3 


41 


-.0308 


.690 


* * 


Special academic programs 


3.13 


3 


49 


-.3558 ** 


. 284 


* * 


Personal attention 


3.32 


3 


33 


— . 0082 


ceo 
. b bo 




Academic f acil ities 


3.32 


3 


50 


-.1825 * 


.255 


* * 


Recreational facilities 


3.18 


3 


37 


-.1901 * 


.422 


* * 


Quality of campus housing 


2.98 


3 


11 


-.1382 


.461 


* * 


Surroundings 


2.86 


2 


.79 


.0720 


.758 


* * 


Campus attractiveness 


3 .45 


3 


45 


.0000 


.620 


* * 


Cost of attendance 


2.54 


2 


.72 


-.1795 


.524 


* * 


Quality of social life 


2.66 


3 


.19 


-.5370 ** 


.397 


* * 


Off-campus opportunities 


2.95 


3 


.20 


-.2500 * 


.422 


* * 


Extracurricular activities 


3.26 


3 


.40 


-.1356 


.261 


* * 


Scale: 1 = Poor/Fair, 2 = Good, 3 = Very 


Good, 4 


= Excellent 






* p < .05 ** p < .01 















Table 4 examines the frequency with which specific images are 
associated with the college when it is rated as CC and as A or B. Since each 
field contains a "1" if the image is marked and a "0" if it is not, the means 
shown represent the percentage of respondents marking each image. Once 
again, the correlations are all high. 



Table 4 

Comparison of Images Marked, CC Only 



College Image 


Meanl 


Mean2 


Difference 


Corr 


Isolated 


. 141 


.148 


-.0074 


.611 ** 


Prestigious 


.541 


.519 


.0222 


.748 ** 


Fun 


. 563 


.474 


.0889 * 


.508 ** 


Intellectual 


. 615 


.652 


-.0370 


.572 ** 


Career-oriented 


. 385 


.430 


-.0444 


.482 ** 


Not well-known 


. 12 6 


.185 


-.0593 


.509 ** 


Comfortable 


.467 


.437 


.0296 


.463 ** 


Back-up school 


. 104 


.185 


-.0815 ** 


.588 ** 


Selective 


. 600 


. 585 


.0148 


.571 ** 


Athletics 


.378 


.437 


-.0593 


.638 ** 


Friendly 


.659 


.644 


.0148 


.543 ** 


Partying 


.259 


.289 


-.0296 


.667 ** 


Average 


.059 


.111 


-.0519 * 


.510 ** 


Challenging 


.726 


.682 


.0444 


.578 ** 



Scale: 1 = Marked, 0 = Not Marked 
* p < .05 ** p < .01 



While there are only three differences that are significant it is in- 
teresting that all of the images that might be perceived as "negative" — 
Isolated, Not Weil-Known, Back-up School , Partying, and Average — were 
marked less often when the college rated was asking the question. 
Conversely, most of the positive images — Prestigious, Fun, Comfortable, 
Selective, Friendly, and Challenging — were marked more often when the 
college rated was the CC. 
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Table 5 compares the mean ratings of the quality of information about 
the college provided by each source of information. For some reason most of 
these diif^rences are enormous. Although the data were rechecked to verify 
these, it is possible that there remains uncaught some error in one of the 
many data transformations that would result in differences of this 
magnitude. All the differences were negative, again meaning that the college 
being rated received higher ratings when it was not the CC. 

Table 5 

Comparison of Quality Ratings, CC Only 
Sources of Information 



Information Source 


Meanl 


Mean 2 


Difference 


Corr 




High school visits 


2.41 


2.98 


-.5690 


* * 


.618 


* * 


College-sponsored meetings 


2.00 


2.67 


-.6727 


* * 


.529 


* * 


College publications 


3.13 


3.27 


-.1374 




.527 


* * 


College videos 


2.10 


2.95 


-.8462 


* * 


.237 




Financial aid communications 


2.39 


2.74 


-.3505 


* * 


.513 


* * 


Campus visit 


3.35 


3.39 


-.0481 




.570 


* * 


On-campus interview 


3.04 


3.29 


-.2500 




.439 


* * 


Post-admissions contact 


3.01 


3. 16 


-.1557 


* 


,654 


* * 


Contact with faculty 


1.99 


2.92 


-.9333 


* * 


.580 


* * 


Contact with coaches 


2.40 


2.97 


-.5714 


* 


.308 




Contact with graduates 


2.17 


2.76 


-.5857 


* * 


.258 


* 


Contact with students 


2.83 


3. 15 


-.3191 


★ 


.369 


* * 



Scale: 1 = Poor/Fair, 2 = Good, 3 = Very Good, 4 = Excellent 
* p < .05 ** p < .01 



Table 6 displays the mean rating of the cost of attending the college. 
Here the cost was perceived to be slightly higher when the rated college was 
asking the question than when it was not. The correlation between the two 
ratings is very high. 

Table 6 

Comparison of Rating of Cost of Attending, CC Only 
Meanl Mean2 Difference Corr 

5.51 5.38 .1376 .866 ** 



Scale: 1 = Very Low, 8 = Very High 
** p < .01 



Discussion 

Although this paper was meant to describe the methodology involved in 
comparing matched ratings rather than the results of the comparisons 
themselves, the preliminary results do have some implications for future 
research. First, in order to make this type of study meaningful the number 
of comparisons must be sufficiently large and representative. The 14 
participants in the 1993 Plus service who are also members of the Consortium 
on the Financing of Higher Education (COFHE) have been asked to participate 
in the 1994 study by supplying SSNs for their respondents. With this group 
of colleges there is a high likelihood that a large number of students will 
have rated more than one college. 

However, the larger the number of cases analyzed, the more likely that 
small numeric differences will be statistically significant. Since it would 
seem to be desirable to find small differences, or even no differences at 
all, in. a study of the consistency of the ratings, the distinction between 
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statistically significant differences and differences that are large enough 
to be important will be critical. It should be noted that some informal 
comparisons of aggregate sets of ratings of CC/competitor pairs, made during 
the field tests of the Plus survey, indicated that there was high consistency 
overall . 

A second point is that the problem of restriction of range will have to 
be dealt with. On the one hand, the range of values is very narrow: 
importance is a three-point scale, quality ratings use a four-point scale, 
the images are dichotomous. The rating of cost uses an eight-point scale, 
and it is noteworthy that the cost ratings for this preliminary study had the 
highest correlation (.866) of all the variables. 

On the other hand, the range of values is likely to be even more 
restricted because the respondents generally rate as A and B colleges that 
are among their top choices. The respondents just don't give very many low 
ratings to colleges they would like to attend. The fact that the respondents 
are forced to rate CC (if they return the questionnaire at all) may explain 
why the ratings of CC tended to be lower than the ratings of the same college 
when it was A or B. Nevertheless, as Table 7 shows, even non-matriculants 
rated CC Very Good or Excellent three-quarters of the time. 



Table 7 

Distribution of Ratings of College Characteristics 



Poor/ Very Excel- N of 

Fair Good Good lent Ratings 

Colleges A and B 3.7% 14.9% 33.8% 47.6% 3025 



CC 



4.0% 



17.7% 



40.9% 



37.3% 



1495 



CC - Matriculants 0.5% 11.0% 36.5% 52.0% 373 

CC - Non-Matrics 5.2% 20.0% 42.4% 32.4% 1122 



The fact that the respondents are forced to rate CC brings up a third 
point, which is that future analyses should differentiate between ratings in- 
volving CC and those of Colleges A and B only. The sign of the difference 
between mean ratings is interpretable when CC is involved, but has no obvious 
meaning for differences between A and B. It may well be that ratings are af- 
fected by when they occur in the admissions cycle and by which questionnaire 
was completed first, but such factors are not measurable for the available 
data. 

In summary, the logistical complexities of dealing with comparisons 
such as those described here are mind-boggling. Nevertheless, because of the 
attraction of asking students to name and rate specific other colleges that 
are strong competitors of one's own (hence the popularity of the ASQ Plus) it 
is worth the effort to try to determine whether the students' ratings are 
consistent. It is to be hoped that the larger study planned for early 1994 
will provide more conclusive data. 
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USING ADMITTED STUDENT QUESTIONNAIRE TO DETERMINE COGNITIVE 
FIT BETWEEN INCOMING STUDENTS AND COLLEGE AND PREDICTING THE 

FUTURE ENROLLMENT BEHAVIOR 



Yun K. Kim, Ph.D. 
Director, Office of Institutional Research 
Goucher College, Baltimore Maryland 

ABSTRACT 

In recent years, the shrinking applicant pool combined with sky 
rocketing tuition and fees forced many small liberal arts colleges to 
think critically about their overall enrollment management. The 
college-fit theory and cognitive consistency and dissonance theories 
suggest "fit" between students and college for the optimum student 
retention. This study used three years of Admitted Student 
Questionnaire data to develop college image types and to learn which 
pre-enrollment image type is most likely to predict the future 
enrollment behavior. The results showed that students who did not 
demonstrate a strong pre-enrollment college image dropped out at a 
higher rate. 

INTRODUCTION 

Traditionally, many small liberal arts colleges enjoyed constant streams of 
high school graduates coming from their feeder schools which helped to maintain a 
healthy and stable enrollment. However, in recent years, the shrinking applicant 
pool combined with sky rocketing tuition and fees forced these colleges to think 
critically about their overall enrollment management. In order to maintain a 
relatively healthy enrollment, these colleges have to compete aggressively with 
larger private universities, as well as, nearby public institutions. The initial action 
taken by many of these colleges was to engage in recruitment campaigns (or 
recruitment marketing). After a decade or so of aggressive recruitment campaigns, 
the market seems to reached its saturation point. Slowly, but seriously many 
administrators and faculty are turning their focus on retention of the students who 
are already enrolled in their respective colleges. Historically, these colieges held 
an elitist view on attrition of their students - "Those students shouldn't be attending 
our college anyway. The college is better off without them!" Unspoken implication 
of this attitude was that the majority (if not all) of student attritions were due to 
students' academic deficiencies. Any individual who acquaints him/her self with the 
findings from numerous retention studies will learn that academic difficulty is just 
only one of multiple reasons for leaving college before earning a degree. Tinto 
(1989) cited five "causes" for student departure - academic difficulty, adjustment, 
goals, uncertainty, and commitments. A national survey on undergraduate 
retention cited financial difficulties, students' accomplishing their goals, other 
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persona! reasons, and poor academic progress as the four major reasons for 
student attrition (Chaney & Farris, 1991). Current reality is that most of small 
liberal arts colleges can no longer operate with an elitist attitude toward student 
attrition, if they hope to maintain academically and fiscally healthy enrollment 
stability. 

Chaney & Farris (1991) found that, in the past five years, 81 percent of 
institutions surveyed had developed programs aimed at increasing retention. 
Probably all of us who are attending this conference are currently engaged in some 
sort of retention study and/or programs. Conventionally used tools such as exit 
survey and interview, nonreturning student survey, informal/formal interview, and 
focus group are all based on an assumption that students are leaving college due 
to "problems" encountered while attending that institution. This ask-and-fix model 
reduces multi-dimensional relationships between students and an institution to one 
or two concrete problems. For example, if 60% of nonreturning students said 
tuition was too high, this model would suggest more money for financial aid as a 
retention strategy. Although financial difficulty is the second most frequently cited 
reason for departure, after controlling for academic ability and motivation, there is 
almost no relationship between income and attrition (Ramist, 1981). i he ask-and- 
fix model completely undermines what Tinto observed (and later supported by 
numerous researchers) as "a longitudinal process of interactions between the 
individual and the academic and social systems of the college" (1975, p.94). This 
author agrees that the ask-and-fix model has served many colleges well and it will 
continue to provide data on why students leave college before actualizing their 
educational goals. However, development of effective campus-wide retention 
strategies require proactive decision making models. The college-fit theory and 
cognitive consistency and dissonance theories offer us additional tools for 
understanding the complexity behind why students leave college without earning a 
degree. 

The college-fit theory suggests that the greater the congruence between the 
values, goals, and attitudes of the students and those of the college, the more 
likely the students are to remain at that school (Taylor & Whetstone, 1983; 
Kalsbeek, D., 1989). Taylor & Whetstone (1983) found that the personal 
characteristics (i.e., values and attitudes) of academically successful men 
engineering students were significantly different from unsuccessful men 
engineering students. These researchers suggested that identifiable persona! 
characteristics of successful students can be described and it can be used to 
assist students in selecting the college setting where they would best "fit." 
Cognitive consistency and dissonance theories help us to understand the causes of 
this observed correlation. These theories explain that, in general, when two 
cognition that are inconsistent with one another will produce discomfort that 
motivates the person to remove (drop out) the inconsistency to bring the cognition 
into harmony (Atkinson, Atkinson et al, 1987). 

Attracting an incoming class with perfect cognitive fit is nearly an impossible 
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task; however, understanding the level of "fit" between the students and the 
college will provide valuable data for total enrollment management and preserving 
educational integrity of small liberal arts colleges. In order to determined the level 
of fit, first we must know students' attitudes about our college before their initial 
enrollment. Admitted Student Questionnaire, a survey instrument designed by the 
College Board, tries to measure what kind of "attitudes" (or images) the accepted 
applicants have on a particular college. In this study, the author investigated a 
possible link between incoming students' pre-enrollment college images with their 
college enrollment behavior. In other words, by knowing one's pre-enrollment 
college images (attitudes about an institution), could we predict his/her enrollment 
persistence. This study attempted to answer two questions: 

1. does a incoming student hold and express identifiable college 
images before his/her initial enrollment? 

2. is one image type "better than other image types, in terms of a 
student's enrollment behavior? 

METHODOLOGY 

Data from college image section of the 1990, 1991, and 1992 Admitted 
Student Questionnaire (ASQ) from a small liberal arts college located in the mid- 
Atlantic region were used for this study. College image section contains 19 
words/phrases, five items are institutional specific that changed every year. Thus, 
the study is based on the fourteen core words/phrases (see Table 1). The 
respondents (admitted applicants) were ask to circle all words or phrases which 
are the most widely-held images of a particular college. Factor analysis was 
performed on the responses collected from 1026 admitted freshman applicants 
(see Table 2). After the factor analysis each individual was grouped into five 
image types. Students who are enrolled for the fall 1993 are classified as active 
students otherwise they were classified as inactive. Chi-Square test was 
porformed to determine statistical significance of the relationship between the 
image type and the matriculated applicants' enrollment behavior. 



Insert Table 1 about here 



Insert Table 2 about here 
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RESULTS 

Factor analysis on the 14 image descriptors identified four factors (see 
Table 3). The Rotated Factor Matrix, shown in Table 3, displays four image 
groupings - academic, environmental (atmosphere), social, and name recognition. 
Image grouping of the admitted applicants and the matriculants are displayed in 
Table 4. As expected, a larger proportion of the matriculants held favorable 
images of the College than their non-enrolling counterparts. Over 80% of the 
matriculants used words like "friendly," "comfortable," "challenging," and 
"intellectual." 



Insert Table 3 about here 



Insert Tabel 4 about here 



As of Fall 1993, 313 (76.16%) of the matriculants are actively enrolled (see 
Table 5). Table 5 shows that the highest attrition (30.56%) has taken place with 
the group of students who did not demonstrate a strong College Image (Unknown 
Group). It was followed by the matriculants who held high academic image of the 
College, which showed an attrition rate of 28.35%, as a group. The matriculants 
who held a positive social image of the College and those who thought the College 
has low name recognition, demonstrated higher retention rates. This relationship 
can be observed in Table 6. It displays the chi-square expected values and the 
chi-square residual values. The top two loses occurred among the students who 
came in with a high academic image (residual = -1 .9) and the students without a 
clear college image (residual ■ -2.4). 



Insert Table 5 about here 
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Insert Table 6 about here 
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DISCUSSIONS AND LIMITATIONS 

This study took a very simplistic approach to examining the interplay 
between the college image and enrollment behavior without considering one's 
academic preparation and abilities, gender, family income, ethnicity, & etc. The 
researcher is aware of limitations in forcing students into one-dimensional image 
type. It is a highly artificial way of understanding the relationship. Most often 
students of this college use adjectives like "friendly," "comfortable," "selective," 
"expensive," "challenging, and "not-well-known" to describe the College. These 
descriptors clearly overlap the factor 1 (academic reputation), factor 2 (Friendly 
Environment) and factor 3 (Name Recognition). Hence any conclusion must be 
made with a great deal of caution. Nevertheless several plausible conclusions 
could be drawn from the results of this study. 

The data suggests that incoming freshmen with somewhat "negative" college 
image (i.e., partying school) persisted at a higher rate than their peers who did not 
demonstrate a strong college image type. The formation of a strong college image 
could be an indicator of students' em otional preparedness for starting college. 
Under this assumption, one could conclude that incoming students with a strong 
image type, regardless of its positivity or negativity, are more sure about their 
educational journeys. And, they are constantly searching for an environment which 
provides a highest cognitive fit between their images and the campus community. 
On the other hand, students who do not demonstrate a strong image type could 
not look for that optimum cognitive fit, because they are unclear ("unknown") about 
their educational goals for attending a college. They are constantly dealing with 
cognitive dissolutions. Most of the time, this cognitive dissolution acts as an 
harmful agent rather than providing the students with creative solutions. A 
challenge for college faculty and administrators is to find out how many of their 
incoming students are in this gray area and establish a program to instill 
"appropriate" educational goals that would be consistent with their particular 
college. A general consensus amo.ig researchers is that freshman orientation can 
be an effective tool for teaching (or indoctrinate) characteristics that are proven to 
be successful for a particular college (Tinto, 1975, 1987, & 1989; Ramist, 1981 ; 
Taylor & Whetstone, 1983; Kalsbeek, 1989; Chaney & Farris, 1991). 

One interesting result of this study is the enrollment behaviors observed 
among the sixteen students who held highly social college image. This researcher 
was expecting to see the greatest attrition among these students. During the last 
five years on-campus social life has been rated extremely low in annually 
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conducted campus environnent survey. This College's students often use the 
word "dead" to describe the, social life in general. !s it possible to assume that 
once a student formulates il jes and image?- , he/she will search the environment 
to validate his/her cognitive, reliefs thus maintaining the cognitive consonance. An 
alternative view is, perhao? students' college images are made up of primary, 
secondary, and even, tertiary images. Furthermore, students are willing to live 
(i.e., stay enrolled) with cognitive dissonances created by secondary and tertiary 
images, but cognitive dissonances created by mis-fit between students' primary 
images and a college is much more difficult to overlook. In extreme cases, these 
cognitive imbalances resul in the eventual departure of a student. 

The college-fit theor/ and cognitive consistency and dissonance theories 
suggest that student retention starts from the moment initial contact between the 
college and the prospective students was made. Therefore, college personnel who 
regularly come in contact with the prospective students must "sell" the college in 
the most realistic way and seek out students who could be best served by that 
institution. Attracting an ever increasing freshman class might not be the most 
healthy thing for the institution nor for the students who must interrupt their 
educational journeys for their mis-matched college choice. After the initial 
enrollment, students must be given an opportunity to learn about the organizational 
culture of an institution. This "cultivation" takes on many forms - freshman 
orientation, First Year Experience, credit bearing college orientation classes, 
academic advising, and counseling are all developed to pass on the knowledge 
which could help the students to be successful in a given college environment. 
Thomas (1990) identified three common features of successful retention efforts. 
According to Thomas, effective retention programs are both comprehensive and 
coordinated. These programs almost always address several areas of students' 
involvement with the academic and social systems of the institution, which requires 
a broad range of college personnel to work cooperatively. Second, successful 
retention programs involve faculty and administrators who consistently establish 
and maintain contact with students - particularly conscious in reaching out to 
freshmen. Third, effective programs usually use a wide range of data (grades. 
SAT scores, demographics, career interests, satisfaction, etc.) and information 
extensively that helps faculty and administrative staff understand more about the 
students and attrition at their college. 

Each student departure, in a minimal sense, represent interruption in the 
student's educational progress, lost of fiscal investment made by the college, and 
lost opportunity for any type of future relationship between the student and the 
college. Therefore understanding the complexity of student retention/attrition is 
critical to preserving educational integrity of many small liberal arts colleges. 
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Table 1 
Word/Phase List 



Isolated 


Back-up school 


Prestigious 


Selective 


Fun 


Athletics 


Intellectual 


Friendly 


Career-Oriented 


Partying 


Not well-known 


Average 


Comfortable 


Challenging 



Table 2 

Number of Admitts and Matriculants Who Participated in ASQ 





All Admitted 


Matriculates 


Frequency 


Percent 


Frequency, 


Percent 


Fall '90 


262 


25.5 


80 


19.40 


Fall '91 


332 


32.4 


153 


37.40 


Fall '92 


432 


42.1 


178 


43.20 


Total 


1026 


100 


411 


100.00 



Table 3 
Factor Analysis 
(Rotated Factor Matrix) 





Factor 1 


Factor 2 


Factor 3 


Factor 4 


Challenging 

Intellectual 

Selective 

Prestigious 

Average 


0.72378 
0.69217 
0.65718 
0.62479 
0.50652 








Friendly 

Comfortable 

Fun 




0.78766 
0.70361 
0.54306 






Partying 
Athletics 






0.67935 




Isolated 

Not-Well-Known 








0.79116 
0.59656 



Table 4 

Factor Distrituion of All Admitts and Matriculants 
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ALL ADMITTS 


MATRICULANTS ! 


IMAGE TYPE 


Frequency 


Percent 


Frequency 


Percent 


Academic Reputation(Factor 1) 


270 


26.3 


122 


29.7 


Friendly Environment (Factor 2) 


419 


40.8 


217 


52.8 


Social rteputation(Factor 3) 


143 


13.9 


16 


3.9 


Name Recognition (Factor 4) 


78 


7.6 


20 


4.9 


Unknown 


116 


11.3 


36 


8.8 


Total 

■ 


1026 




411 





3h ^ 



Tables 

Number of Actively Enrolled Student 





ORIGINAL 


ENROLLED 




IMAGE TYPE 


ENROLLMENT 


(as of Fall '93) 


PERCENT 


Academic Reputation (Factor 1) 


127 


91 


71.65% 


Friendly Environment(Factor 2) 


217 


166 


76.50% 


Social Reputation (Factor 3) 


16 


13 


81 .25% 


Name Recognition (Factor 4) 


20 


18 


90.00% 


Unknown 


36 


25 


69.44% 


Total 


411 


313 


76.16% 



Table 6 

Enrollment Status by Image Type 



Count 








Chi-Sq. Exp. Value 


ACTIVE 


INACTIVE 


RAW 


Chi-Sq. Resid. Value 


STUDEr 


* TUDENT 


TOTAL 




91 


31 


122 


Academic Reputation 


92.9 


29.1 






-1.9 


1.9 






166 


51 


217 


Friendly Environment 


165.3 


51.7 






-0.7 


0.7 






13 


3 


16 


Social Reputation 


12.2 


3.8 






-0.8 


0.8 






18 


2 


20 


Name recognition 


15.2 


4.8 






2.8 


2.8 






25 


11 


36 


Unknown 


27.4 


8.6 






-2.4 


2.4 






313 


98 


411 




(76.2%) 


(23.8%) 


(100.0% 


CHI-SQUARE 


D.F. 


SIGNIFICANCE 


3.41123 


4 


0.4915 





Missing observations = 36 
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FACULTY SALARY EQUITY - COMPARISON OF TWO METHODOLOGIES 

Denis e A. Krallman 
Institutional Research Analyst: Budgeting, Planning & Analysis 

Miami University - Oxford 



Introduction 

Salary equity studies have been conducted at this university 
since 1978. Over this fourteen year span, the techniques used have 
been reviewed and modified to meet the ever growing demand for 
accuracy in the investigation of bias in salaries. This year similar 
procedures were used in order to maintain the continuity and 
replicabili ty of the past. In addition, compa-ratio analysis, a 
corporate procedure recently applied to higher education, was 
introduced to compare the results found in the standard regression 
analysis . 

I. Regression Analysis 
Methodology 

The traditional stepwise regression analysis was used to identify 
the attributes related to professional experience and qualif ications 
that are correlated with the university faculty's current salaries. 
Although this procedure has drawbacks because of the lack of "human" 
input into the determination of which variables should be added, it 
has been used locally for a number of years and, therefore, to 
maintain consistency it was again used this year. 

Data were gathered on 860 1992-93 full-time university faculty, 
using the same criteria for inclusion as in the AAUP and IPEDS salary 
surveys. In order to optimize group sizes, lecturers and senior 
instructors were grouped with assistant professors, assistant 
instructors were grouped with instructors and the faculty at the two 
branch campuses were grouped together as regional campus faculty. The 
following data elements were used in the regression equations for 
predicting 1992-93 faculty salaries: academic rank, years of 
university service, age, years in current rank at the university, 
years since completion of highest degree, possession of doctorate 
(dummy variable with 0 = no doctorate, 1 = doctorate or other higher 
degree) and relative market value of the individual's discipline. 
This disciplinary market factor is from the NASULGC 1991-92 Faculty 
Salary Survey by Discipline conducted by the Office of Institutional 
Research at Oklahoma State University. It is a measure of current 
relative disciplinary market value defined as the national ratio of 
(the average salary paid to faculty at a given rank in a specific 
discipline) to (the average salary paid to faculty at that rank in all 
disciplines combined) . This is used in place of department, which is 
included in many salary equity studies, to identify differences in 
market value or worth. 

Two models were used within stepwise regression to determine 
salary equity at each individual rank and then again using the four 
ranks as dummy variables. They were: all white main campus males, 
and all faculty. From these analyses, the residuals (actual minus 
predicted salary) were tested for significant differences on gender 
and campus (main versus regional) using analysis of variance (ANOVA) . 
Similar procedures were used to test for ethnic salary bias, and for 
divisional differences. 
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Analysis and Results 
Gender Equity 

The first series of analyses - using both models - looked at each 
rank independently. The model using white main campus male faculty as 
the basis for building an equation for predicting salary was run first 
to determine the existence of gender inequities in salaries at each 
rank This is the standard process for detecting existing bias . 
After the equations were established with the predictor variables 
(Table 1), a 2-way ANOVA was run on the residual salaries (Actual 
salary minus Predicted salary) to test for statistically significant 
differences for gender and campus (main vs regional) . At each of the 
four ranks, there were no statistically significant differences 1) 
between males and females with respect to their residual salaries, 2) 
between the main campus and the regional campus and 3) interactions 
between gender and campus. 

The second model, using all faculty as opposed to white main 
campus males, is based on the assumption that there is no bias in 
salary (see above ) and is typically the primary model used when the 
purpose of a study is to explain or predict salary levels. After the 
stepwise regression analysis was run, equations were established for 
each rank (Table 2) . When a 2-way ANOVA (gender by campus) was run, 
no statistically significant differences were found. Tables were 
supplied to the Provost that included information for each individual 
faculty with respect to their salary, residual from predicted salary 
and those who fell 1.0 to 2.5 standard deviations above or below the 
mean residual of their respective rank. 

The second series of analyses - run on both models - used each 
rank as a dummy variable (e.g., Professor: 1 = Yes 0 = No) . As 
before, the white main campus male model was run first to identify any 
bias in salaries. After the residuals were calculated using the 
equation of statistically significant predictors (Table 3), a 2-way 
ANOVA was run (gender by campus) for each rank. The only 

statistically significant difference found on the residuals was a 
gender by campus interactive effect at the Associate Professor level. 
The mean residuals show that the male Associate Professors at the 
regional campuses have much lower actual salaries with respect to 
their predicted salaries than do any of the other three groups (-2,912 
vs 60, -630 and 1,077). This large difference, coupled with the 
female Associate Professors at the regional campuses having the 
largest positive mean residual ( 1 , 077 ) , explains the statistically 
significant interaction effect. 

In the all- faculty model using rank as a dummy variable, an 
equation almost identical to the white main campus male model was 
developed. In the 2-way ANOVA of the residuals, a statistically 
signific difference for Associate Professors was again found in the 
gender L u campus interaction. The mean residuals show the same 
pattern as in the previous model , wi th male Associate Professors on 
the regional campuses having a markedly larger negative mean residual, 
and the regional campuses female Associate Professors with a large 
positive mean residual . 
Ethnic Equity 

The same procedures were used to test for ethnic salary 
inequities as were used for testing for gender bias in salaries. 
Because of the small number of full-time faculty in the various ethnic 
groups, all minority faculty were grouped together for the initial 
regression analyses. Then, if any statistically significant 

differences were found, separate analyses would be run on those means 
to identify which groups were actually contributing to the 
di r ferences . 
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Table 1 

Stepwise Regression Equations for Salary Prediction 



White Main Campus Male Model 



Regression Coefficients 



Professor 



Associate 



Assistant 



instructor 



Possible Predictions 


b 


R-Sq 


b 


R-Sq 


b 


R-Sq 


b 


R-Sq 


Age 






188.13 


0.66 










Years of Service 


-874.95 


0.46 






563.65 


0.82 






Years in Rank 


994.72 


0.25 


358.34 


0.54 






693.55 


0.57 


Years Since Hichest Degree 


693.49 


0.63 






-188.55 


0.83 






Pnc^pqq HnnrnratP 


Disciplinary Market Factor 


327.35 


0.56 


345.99 


0.44 


333.75 


0.73 






Constant 


20989.25 




867.01 




2930.06 




26771.12 






Overall R-Sq 




0.63 




0.66 




0.83 




0.57 



Table 2 

Stepwise Regression Equations for Salary Prediction 
AI! Faculty Model 



Regression Coefficients 



Professor 



Associate 



Assistant 



Instructor 



Possible Predictions 


b 


R-Sq 


b 


R-Sq 


b 


R-Sq 


b R-Sq 


Age ! 
















" Years of Service 


-819.33 


0.46 


174.87 


0.63 


458.45 


0.75 




■ Years in Rank 


1045.89 


0.27 


344.89 


0.61 






658,52 0.22 


Years Since Highest Degree 


600.23 


0.63 












Possess Doctorate 
















Disciplinary Market Factor 


327.16 


0.57 


328.04 


0.41 


324.96 


0.55 


125.45 0.32 


Constant 


21560.43 




8991.61 




3190.63 




13957.09 




Overall R-Sq 




0.63 




0.63 




0.75 


0.32 



Notes: 

1 . A blank coefficient ( — ) entry indicates that the preaictor did not contribute to the regression equation. 

2. "b" = Unstandardized regression coefficient. 

3. "R-Sq" = Amount of variance in salary accounted for by the predictor as loaded into the regression equation. 

4. "Possess Doctorate" variable was dummy coded. T meant possess doctorate, "0" meant no doctorate. 

5. Disciplinary Market Factor" taken from 1 991-92 Faculty Salary by Discipline Study of NASULGC Schools by 
Oklahoma State University. 

6. "Overall R-Sq" = The total varience of salary accounted for by regression equation. 
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Table 3 

Stepwise Regression Equations for Salary Prediction 



Two Models Using Rank as a Dummy Variable 




Repression Coefficients 






White Male Main Cmp Faculty, 


All Faculty 




Possible Predictions 


b 


R-Sq ! 


b 


R-Sq 


Academic Status 




I 






Professor 


2221 1 .61 


0.47 


21525.40 


0.49 


Associate 


8169.09 


0.78 


7415.99 


0.61 


Assistant 










Instructor 


-9284.57 


0.80 


-9561 .66 


0.82 




Age 


172.48 


0.83 


131 .65 


0.84 


Years of Service 


-520.26 


0.82 


-385.67 


0.83 


Years in Rank 


850.22 


0.72 


782.42 


0.79 


Years Since Highest Degree 


171.17 


0.83 


128.64 


0.84 


Possess Doctorate 










Disciplinary Market Factor 


324.85 


0.59 


314.02 


0.70 



Constant -3692.03 -S54.25 



Table 4 

Stepwise Regression Equations for Salary Prediction 
White Faculty Model (Ethnic Bias) 

Regression Coefficients 



Professor Associate Assistant Instructor 



Possible Predictions 


b 


R-Sq 


b 


R-Sq 


b 


R-Sq 


b R-Sq 


Age 
















Years of Service 


-803.12 


0.46 


181.77 


0.61 ' 


482.31 


0.78 




Years in Rank 


957.14 


0.27 


319.17 


0.60 






668.40 0.24 ' 


Years Since Highest Degree 


697.14 


0.63. 












Possess Doctorate 
















Disciplinary Market Factor 


336.04 


0.57 


322.55 


0.41 


328.75 


0.56 


135.64 0.34 


Constant 


19005.15 




9567.46 




2457.04 




12858.03 




Overall R-Sq 




0.63 




0.61 




0.78 


0.34 



Notes: 

1 . A blank coefficient ( — ) entry indicates that the predictor did not contribute to the regression equation. 

2. "b* = Unstandardized regression coefficient. 

3. "R-Sq" = Amount of variance in salary accounied for by the predictor as loaded into the regression equation. 

4. "Possess Doctorate" variable was dummy coded. "1" meant possess doctorate, "0" meant no doctorate. 

5. Disciplinary Market Factor* taken from 1991-92 Faculty Salary by Discipline Study of NASULGC Schools by 
Oklahoma State University. 

6. "Overall R-Sq" = The total varience of salary accounted for by regression equation. 
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The first model was developed using all white faculty and was run 
for each of the four ranks (Table 4) . When ANOVAs were run on the 
residuals comparing white faculty to minority faculty, the only 
statistically significant difference was found at the Assistant 
Professor level. Here minority faculty had a mean residual of over 
$1,400 higher than white faculty. 

In the second model, using the all faculty model, an ANOVA was 
run on the residuals comparing minority to white faculty and as 
expected (since there was a significance in the previous model) , there 
was a statistically significant difference at the Assistant Professor 
level (white = -209; minority = 1,147). Because of this significant 
difference, T- tests were than run for each ethnic group against the 
remaining faculty to identify where the differences at the Assistant 
Professor level were actually occurring. The results of tliese T-tests 
show that the difference is occurring in the black assistant professor 
salaries, that is, black Assistant Professors have statistically 
significant higher positive residuals (actual minus predicted) than 
the remaining Miami Assistant Professors. A further breakdown shows 
that this difference can be tied to the black female Assistant 
Professors . 
Divisional Equity 

Using the equations defined by the "all faculty" model, 
residuals were analyzed for divisional differences using ANOVA. 
Statistically significant differences were found between divisions at 
all four ranks (Table 5) . 



Table 5 

ANOVA of Divisional Differences on 
Mean Residual by Rank 



Rank 


F~Statistic 


p-value 








Professor 


3 .134 


. 009 


Associate Prof 


3 .911 


. 002 


Assistant Prof 


2 .458 


.034 


Instructor 


3 .997 


.003 



In order to determine which divisions were contributing to these 
differences, T-tests were run for each of the ranks, comparing each 
division's mean residuals for faculty at these ranks to the 
remaining university faculty at the same rank. The tests found 
that for Professors, the statistically significant divisional 
difference can be attributed to School ' B' where Professors in this 
school have mean residual almost $3,700 greater than the remaining 
university Professors and to School ' D' whose Professors have a mean 
residual of nearly $2,600 less than the remaining university 
professors . 

At the Associate Professor level, the statistically 
significant difference is found to lie in two divisions - School ' F ' 
where the mean residual for Associate Professors is over $3,600 
greater than the mean for all other Associate Professors and in School 
'A' with a lower mean residual (difference of $1,000) than the 
remainder of the faculty at this rank. For Assistant Professors, the 
difference was found in School 'A' with a lower mean residual from the 
rest of the faculty and statistically significant T-value . 
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IX » Compa-Ratio Analysis 

Methodology 

In a recent study, Bereman and Scott (1991 ) x introduced compa- 
ratio analysis, a tested corporate practice for reviewing salary and 
gender bias, to higher education. The compa-ratio is the "ratio of an 
actual pay rate (numerator) to the midpoint for the respective pay 
grade (denominator)" (Bereman and Scott, 1991) . For example, a compa- 
ratio for an individual of less than 1.0 says that he/she is being 
paid below the average and a compa-ratio for an individual that is 
greater than 1.0 indicates that he/she receives compensation above the 
pay-grade average. 

In industry, the compa-:;atio has been used for cost control, 
merit pay decisions , to monitor organizational unit compliance with 
overall pay policy, to audit wage changes and to examine the effect of 
turnover on wage costs. Finally, it has been "applied to the problems 
of differential salary treatment of selected groups of employees in 
conjunction with multiple regression techniques. After separating 
employees by such demographics as gender, race or age group, a compa- 
ratio analysis could provide a first indication that there is 
potential bias for or against certain groups of employees" (Bereman & 
Scott, 1991) . 

In their analysis, applying this theory to higher education, 
Bereman and Scott developed a compa-ratio based on faculty rank and 
discipline. This was achieved by dividing each faculty member's 
salary by the NASULGC salary average for his/her rank and discipline. 2 
This ratio was then used to identify faculty who were above or below 
the national average- Their findings indicated that the compa-ratio 
is readily adaptable to the higher education environment and easily 
understood by administrators and faculty. "It permitted the 

institution to examine all salaries in the context of the national 
academic labor market, thereby mitigating internal discriminatory 
influences while assessing relative competitiveness across 
disciplines" (Bereman & Scott, 1991). The key to the acceptance of 
compa-ratio in salary equity analyses rests on the acceptance of rank 
and discipline as appropriate predictors of salary. In summary, the 
authors pointed to possible applications of the compa-ratio technique: 
1 . Evaluate an institutional salary structure internally , i f one 

identifies midpoints for the various ranks as the mean of the 

salaries in each. This would enable higher education 

administrators to: 

identify structural problems, such as compression and 
inversion. 

to gauge how well the rewards system supports institutional 
goals . 

to compare salary levels across academic units, 
to estimate the costs of cor receive measures . 
2. In conjunction with external salary data, the compa-ratio could 
compare an institution's pay scale to the larger academic labor 



1 Nancy A. Bereman and Joyce A. Scott, "Using the Compa-Ratio 
to Detect Gender Bias in Faculty Salaries" Journal of Higher 
Education , 62(1991), 556-569. 

2 This average salary is found in the NASULGC 1991-92 Faculty 
Salary Survey bv Discipline conducted by the Office of Institutional 
Research at Oklahoma State University. 
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market. This would allow the university to: 

measure relative competitiveness in discipline areas, 
measure the cost of enhanced competi tiveness . 
3. In the context of salary biases, the compa-ratio could serve as 

a moni toring device , a means by which institutional progress 

toward salary equity would be checked and understood. 
Analysis & Results 

Two different methodologies were utilized using the compa-ratio 
concept to evaluate salaries at the university. First, simple 
comparisons were made between male/ female , main/ regional campus , 
minority/non-minority and divisional compa-ratios at each rank. 
Second, new regression analyses were conducted eliminating the 
disciplinary market factor and rank as components of the equation and 
replacing faculty salary with individual compa-ratios to determine the 
existence of gender biases. 

Method 1 

Using the individual faculty compa-ratio, subgroup means were 
calculated by averaging individual ratios rather than dividing the 
total salaries by total NASULGC rank/discipline averages. Table 6 
shows the means by various subgroups that identify how the 
university's faculty salaries compare to the national average for 
particular ranks and other subgroups - gender, campus, ethnicity and 
division . 

Looking at the compa-ratio by gender, it is apparent that overall 
both males and females at all ranks have salaries that are comparable 
to or even above the average of the national salaries (taking into 
account discipline) . When tested for difference between the means 
there is a statistically significant difference at the associate 
professor level, with the male compa-ratio higher than the female 
compa-ratio . 

There was a statistically significant campus difference at the 
associate professor level, but there were no statistically significant 
minority/non-minority differences at any rank. Finally, when a 2-way 
ANOVA was run on the compa-ratio of gender by campus, only gender 
differences were statistically significant (at the Associate Professor 
level)- campus differences became statistically insignificant. That 
is, when both gender and campus location are considered together, 
gender alone accounts for any statistically significant differences in 
the compa-ratio at the Associate Professor level. 

Method 2 

In this method, the compa-ratio is substituted for salary in the 
regression analyses and disciplinary market factor and rank are 
removed as predictors. After a regression equation was developed, 
ANOVAs were run on the residuals to look for significant differences 
at each rank with respect to gender and campus and on the faculty as 
a whole. The regression results are shown in Table 7. 

The first regression was run on white main campus male faculty 
to determine the presence of gender inequities at each rank and to 
parallel the standard regression models. The variables included in 
the stepwise regression were age, years of service at the university, 
years in current rank , years since highest degree received and the 
possession of a doctorate. 

When 2-way ANOVAs were run at each rank on the compa-ratio 
residuals created from the whi te main campus male regression model , 
significant F-statistics were found for Professors by gender, for 
Associate Professors by campus level and for Assistant professors in 
the gender by campus interaction. For both male and female 
Professors, their mean compa-ratio residuals were negative (actual 
smaller than predicted) , however, male Professors displayed a larger 
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negative value (-.026) implying that they have a smaller actual compa- 
ratio as compared to their predicted than do their female counterparts 
(-.021) . That is, female Professors have a mean compa-ratio closer to 
predicted than do male professors. 

At the Associate Professor level, the statistically significant 
difference is caused by the regional campus Associate Professors 
having a mean residual of -.023 compared to the main campus faculty 
where their mean residual was .029. That is, the compa-ratio 
predicted for Associate Professors at the regional campuses was higher 
than their actual compa-ratios and the resulting mean residuals were 
markedly different from their main campus counterparts. 

At the Assistant Professor level the mean residuals of the 
interaction of gender and campus show that regional campus female 
faculty have the lowest mean residual while the regional campus male 
faculty have the highest mean residual. That is, female Assistant 
Professors on the regional campuses have compa-ratios below the compa- 
ratio as predicted by the regression equation, and regional campus 
male Assistant Professors have compa-ratios above their predicted 
ratios . 

Using the all-faculty model, residuals were calculated from the 
regression equation (Table 5) and 2-way ANOVAs were then run on these 
residuals using gender and campus. The results from the ANOVA's were 
of the same type as those found in the white main campus model , a 
gender difference for Professors, a campus difference at the Associate 
Professor rank and a significant gender by campus interaction at the 
Assistant Professor level. 
Summary 

Listed below are some of the more important highl ight s of the 
analyses discussed above. These do not encompass all of the findings 
but do identify those that appear to have the greatest consequences 
with respect to the salary equity issue at this university. 



Using the traditional stepwise regression model, no gender 
biases were found at any of the four ranks studied. 
When rank was added as a predictor in the regression model 
(as dummy variables) , a gender by campus interaction effect 
was found at the Associate Professor level. 

In the investigation of ethnicity bias in salaries, it was 
found that black female Assistant Professors are paid more 
than other assistant professors wi th respect to their 
predicted salaries . 

In the analysis of divisional differences in the salary 
equity regression model, School had larger positive 

residuals than the other divisions at the Professor level . 
Divisional di f f erences were also found at the Associate 
Professor rank and were attributable to School 'F', where 
the Associate Professors in this school have larger 
positive residuals than the other divisions and to School 
'A' with a negative mean residual. However, no particular 
component of the college could be shown to be the major 
contributor to this difference. 

Market factor was utilized in different ways in this study. 
As a predictor of salary in the regression models, 
disciplinary market factor loaded as a contributing 
variable at all four levels. 

Compa-ratio analysis, a new portion to this year's 
salary equity study, used disciplinary salaries 
(which define market value) as part of the component 
to take the place of salary in the regress ion model . 
In the compa-ratio analysis , looking at gender and campus 
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Table 6 
Mean Compa-Ratios 
By Rank and Selected Subgroups 



Group 


Total 
Mean 


N 


Professor 
Mean 


N 


As$oci3te 
Mean 


N 


Assistant 
Mean 


N 


Instructor 
Mean 


N 


Total 


:.03 


860 


1.01 


265 


1.05 


277 


1.03 


247 


1.03 


71 


Male 


1 .03 


597 


1.01 


238 


1.06 


197 


J. 03 


126 


1 06 


36 


Female 


l!o£ 


263 


1.00 


27 


1.02 


80 


1.03 


121 


1.01 


35 


Main 


1.03 


756 


1.01 


255 


1.05 


242 


1.02 


198 


1.04 


61 


Regional 


1.02 


104 


0.97 


10 


1.00 


35 


1.05 


49 


1.00 


10 


School 'A* 


1.03 


456 


1.01 


156 


1.07 


140 


1.02 


131 


1.00 


29 


School *3" 


1.11 


115 


1.10 


32 


1.07 


33 


1.09 


36 


1.25 


14 


School C 


0.96 


145 


0.95 


38 


0.97 


49 


0.99 


37 


0.87 


21 


School 'D' 


1.03 


76 


1.00 


25 


1.05 


27 


1.03 


22 


1.17 


2 


Schcol "E" 


1.01 


55 


0.97 


11 


1.04 


23 


0.96 


17 


1.19 


4 


School 'F 


1.11 


13 


0.93 


3 


1.13 


5 


1.14 


4 


1.46 


1 


Minority 


1.03 


72 


1.03 


11 


1.03 


17 


1.03 


38 


1.00 


6 


Non-Minonty 


1.03 


788 


1.01 


254 


1.05 


260 


1.02 


209 


1.04 


65 
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Table 7 

Stepwise Regression Equations 
for 

Compa-Ratio Prediction 



Regression Coefficients 
• White Male Main Cmp Faculty All Faculty Model 



Possible Predictions 


b 


R-Sq 


b 


R-Sq 


Age 


0.003 


0.35 






Years of Service 


-0.008 


0.33 


-0.006 


0.29' 


; Years in Current Rank 


0.017 


0.28 


0.017 


0.26' 


Years Since Highest Degree 






0.002 


0.30 


Possess Doctorate 


-0.053 


0.34 


•0.029 


0.30 


. Constant 


0.882 




0.973 






Overall R-Sq 




0.35 




0.30 



Notes: 

1. A blank coefficient ( — ) entry indie ies that the predictor did not contribute to 
the regression equation. 

2. "b" = Unstandardized regression coefficient. 

3. "R-Sq" = Amount of variance in salary accounted for by the predictor as loaded into 
the regression equation. 

4. "Possess Doctorate" vanable was dummy coded. "1" meant possess doctorate, 
"0" meant no doctorate. 

5. Disciplinary Market Factor" taken from 1991-92 Faculty Salary by Discipline Study of 
NASULGC Schools by Oklahoma State University. 

6. "Overall R-Sq" = The total varience of salary accounted for by regression equation. 
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means independently showed statistically significant 
differences for both at the Associate Professor level. 
However, when a 2 -way ANOVA was run, only gender appeared 
statistically significant. 

When a stepwise regression analysis, parallel to the 
original regression models, was run using compa-ratio in 
place of salary, differences were found that were not found 
in the salary regression models. In this new regression, 
statistically significant gender by campus interactions 
were found for Assistant Professors in the " whi te main 
campus male " model and the "all faculty model " . In 
addition, both models showed a statistically significant 
campus difference at the Associate Professor level, and a 
statistically significant gender difference for Professors. 



Cone 1 us ion 

The traditional salary equity analyses conducted this year 
continue to reinforce the knowledge that Miami University does not 
adversely discriminate on the basis of gender or ethnicity with 
respect to faculty salary. However, when looking e.t the regression 
model with rank as a dummy variable, a gender by campus interaction 
effect was found at the Associate Professor level. 

When using compa-ratio as the basis for determining salary 
equity, simple tests of the mean showed a gender difference at the 
Associate Professor level. When using a regression model, replacing 
salary with compa-ratio, additional differences were found - gender by 
campus interaction at the Assistant Professor level, a campus 
difference at the Associate Professor level and a gender difference at 
the Professor level. However, these differences do not necessarily 
reflect an inequity in the direction most often assumed in faculty 
salaries. In fact, the opposite is true. That is, the average male 
faculty compa-ratio is lower than the female (or in the case of 
regression analyses, the mean residual) at both the Associate 
Professor and Professor levels. The campus difference at the 
Associate Professor level in the regression analyses is reflective of 
the results from the standard regression models. Finally, at the 
Assistant Professor level, the interaction of campus and gender can be 
explained by the combining of Senior Instructor (found largely on the 
regional campuses) with Assistant Professor and the phenomena found at 
this university - tenured Assistant Professors (once again, more 
readily found on the regional campuses) . 

These studies supply us with the ability to identify those 
faculty, male and female, whose actual salaries fall significantly 
above and below what is predicted for them given criteria such as 
years of service, years in current rank and the discipline in which 
they teach. With this information, the administration can continue to 
address inequities concentrating on those areas where statistically 
significant differences have been identified. 




SALARY COMPRESSION ANALYSIS 
Denise A. Krallman 
Institutional Research Analyst: Budgeting, Planning & Analysis 

Miami University 



Int roducti on 

The general assumption within higher education is that since 
"faculty capability is basically tied to professional maturity, then 
internal average salary structures should exhibit patterns that 
increase with professional maturity (measured using rank and tenure in 
rank) 1,1 . Salary compression occurs when this link between professional 
maturity and salary are disproportionate . Many factors can cause 
salary compression, some of them very appropriate. However, this does 
not help the morale and feelings of alienation of those who are 
experiencing the compressed salaries . 
Methodology 

This study will approach the concept of salary compression from 
two vastly different methods. The first, a continuation of work done 
at this university over the past two years, focuses on faculty with 
doctoral degrees and uses years-since-doctorate as the measurement of 
maturity. Salary growth was then defined in relation to what new 
assistant professors without any post-doctoral experience were paid at 
this university in 1992-93, with adjustments for disciplinary market 
factors . 

The second method follows the procedure described by Snyder 
(1992) where salary comparisons within ranks were made using 
standardized average salaries and time-in-rank for divisions and the 
regional campuses . 
'Home' Meth od 

The average 1992-93 salary of the new assistant professors who 
were awarded doctorates in 1992-93 and who began university services 
during the academic year 1992-93 was $33,300 and the mean disciplinary 
market factor for the newly hired assistant professors was 93.17 : . 
Therefore, by knowing the mean disciplinary market factor for the 
newly hired assistant professors, it is possible to estimate the 1992- 
93 starting salary for a new assistant professor for each department 
whether or not a new person was actually hired in that department for 
the current academic year. 

At both the main and regional campuses, for each full-time 
faculty member who has a doctorate, a projected salary was computed as 
the present value of the estimated 1992-93 new assistant professor's 
salary in the faculty member's discipline compounded at a given 



: Julie K. Snyder, Gerald W. McLaughlin & James R. Montgomery, 
"Diagnosing and Dealing with Salary Compression" , Researc h i n Higher 
Education , 33 (1992), 113-124. 

"This market factor is from the NASULGC Faculty Salary Survey 
by Discipline conducted by the Office of Institutional Research at 
Oklahoma State University and is the ratio of the average salary of 
new assistant professors in a given discipline to the average salary 
of new assistant professors in all disciplines combined. 
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uniform rate for the number of years since the doctoral degree was 
awarded. Actual salaries were then compared to the projected salaries 
at five assumed annual growth rates ranging from 1.0% to 3.0% in half- 
percent increments i_teed in previous studies. 

Figure 1 shows trie percentage of faculty in each post-doctorate 
cohort group (in 7 year increments by number of years since completion 
of doctorate ) whose present salaries are below their pro j ected 
salaries assuming uniform growth rates of 1.0%, 1.5%, 2.0%, 2.5% and 
3.0%. Seven year increments were chosen for this study to coincide 
with the length of time an assistant professor must have in service 
before tenure. Interpretation of this graph yields the following 
observations : 

1. Faculty with 7 years or less since their doctorate had 60 
percent or more with actual salaries below projected for 
four of the five growth rates, the exception being at the 
1.0% growth rate whe^e the percentage of faculty with 
actual salaries below projected is slightly below 60 
percent . 

2. Because of a relatively small N for those with 29 or more 
years since their doctorate (N = 33), the data should be 
considered carefully before drawing any conclusions about 
longevity and compression. 

3. At 2.5% growth rate, nearly 60 percent of all faculty with 
doctorates have actual salaries below their projected 
salaries regardless of the number of years since their 
doctorate. At the 2.0% growth rate, this percentage drops 
to 43. This would advance the theory of a normal growth 
rate of between 2 . 0% and 2 . 5% where 50 percent of the 
faculty fall above/below their projected salaries. 

Table 1 shows the percentage of faculty in each academic division 
whose actual salaries fall below the salaries projected at each of the 
five growth rates. From this table it can be seen that School 1 C' has 
the most serious problem with salary compression, followed by School. 
'A' . For each growth rate, these two divisions consistently display 
the largest percentages of faculty with actual salaries below their 
projected growth salaries. When looking at divisional differences 
using the time intervals discussed above, the following observations 
can be made (see Table 2) : 

1. School ' C plays a large role in each time interval, 
accounting for a considerable portion of those faculty who 
do not meet or exceed their projected salar'es at the 
various growth rates. That is, compression affects the 
faculty within the school in all intervals to a greater 
extent than the other divisions. 
2 . In the " 7 Years or Less Since Doctorate " category, School 
'A' faculty also contribute heavily to the overall percent 
with actual salaries below their projected growth salaries. 
In the other time intervals, School 'A' faculty are more 
cons is tent with the faculty in the other divisions . 
3. The doctoral faculty in School 'B' are less affected by 
compression at all time intervals for all five growth rates 
when compared to Schools 'A' and ' C . 
When looking at the projected growth salaries by rank and 
division at each of thu five growth rates (Table 3), the doctoral 
faculty within School 'C continue to display the most significant 
impact because of compression . For example , using the 2 . 0% growth 
rate, within School 'C 63.9 percent of the 36 professors, 90.2 
percent of the 41 associate professors, and 83.3 percent of the 30 
assistant professors had salaries below 7 their projected growth 
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Table 1 



Percent of Doctoral Faculty Whose Actual Salaries 
Are Below Projected Growth Salaries by Academic School 



Percentage of Faculty 





No. of 




At Each Growth Rate 






School 


Faculty 


1 .0% 


1.5% 




2.5% 


3.0% 


School 'A' 


396 


1S.9 


27.0 


40.9 


60.6 


78.0 


School 'B* 


95 


7.4 


8.4 


20.0 


34.7 


53.7 


School 'C 


107 


56.1 


66.4 


79.4 


87.9 


94.4 


School 'D' 


26 


11.5 


15.4 


15.4 


30.8 


46.2 


School 'E' 


24 


8.3 


20.8 


45.8 


58.3 


62.5 


School *F 


12 








16.7 


25,0 


TOTAL DOCTORAL FACULTY/AVG 


660 


22.4 


29.7 


42.7 


59.2 


74.4 



Tab* 2 

Percent of Doctoral Faculty Whose Actual Salaries 
Are Below Proiected Growth Salanes By 
Academx; School and Years Since Doctorate 



Percentage of Faculty 





No. of 




At Each Growth Rate 






School Bv Years 


Faculty 


122a 




2.0% 


2 5% 


3.0% 


7 Years or Less 


157 


57.3 






60.8 


712 


School 'A' 


83 


63.9 


72.3 


72.3 


74.7 


79.5 


School 'B' 


21 


23.8 


23.8 


23.8 


33.3 


42.9 


School 'C 


35 


82.9 


85.7 


91.4 


94.3 


97.1 


School "D" 


7 


42.9 


57.1 


57.1 


71.4 


71.4 


School T 


7 


0.0 


0.0 


00 


14.3 


14.3 


School 'F 


4 


0.0 


00 


0.0 


0.0 


0.0 


8io 14 Years 


150 


21.3 


33.3 


4U 


58.7 


72.0 


School A 


80 


16.3 


31.3 


37.5 


SZ5 


77.5 


School *B* 


25 


8.0 


12.0 


24.0 


32.0 


44.0 


School C' 


29 


55.2 


69.0 


79.3 


$62. 


98.6 


School 'D' 


12 


0.0 


0.0 


0.0 


16.7 


33.3 


School 'E' 


2 


0.0 


5O.0 


100.0 


100.0 


100.0 


School 'F 


2 


50.0 


5O.0 


50.0 


50.0 


50.0 


15 to 21 Years 


158 


10.7 


17.9 


393 


57.1 


73.6 


School 'A' 


95 


5.3 


12.6 


35.8 


53.7 


80.0 


School 'B' 


27 


0.0 


0.0 


185 


40.7 


63.0 


School *C 


29 


37.9 


51.7 


75.9 


89.7 


98.6 


School 'D' 


5 


0.0 


0.0 


0.0 


20.0 


40,0 


School 'ET 


9 


22.2 


33.3 


55.6 


77.8 


88.9 


School "F 


3 


00 


0.0 


0.0 


0.0 


33.3 


22 to 20 Years 


133 


3.8 


7.5 


28.6 


55.8 


78.9 


School 'A' 


102 


2.9 


5.9 


26.5 


5S.9 


80.4 


School 'B' 


16 


0.0 


0.0 


18.8 


37.5 


68.8 


School 'C 


8 


25.0 


50.0 


625 


87.5 


100.0 


School 'D' 


1 


0.0 


00 


0.0 


0.0 


0.0 


School T 


3 


0.0 


0.0 


100.0 


100.0 


100.0 


School 'F 


3 


0.0 


0.0 


0.0 


33.3 


33.3 


29 Years Of MOfe 


33 


9.1 


21 -2 


455 


75J 


93,9 


School 'A' 


25 


4.0 


16.0 


44.0 


80.0 


92.0 


School 'B* 


3 


0.0 


0.0 


0.0 


33.3 


100.0 


School 'C 


3 


66.7 


66.7 


100.0 


100.0 


100.0 


School 'D' 




0.0 


0.0 


0.0 


0.0 


100.0 


School 'E- 




00 


100.0 


100.0 


100.0 


100.0 


School 'F 
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Table 3 



Percent of Doctoral Faculty Whose Actual Salaries 
Are Below Projected Growth Salaries By 
Academic School and Rank 



Percentage of Faculty 





No. of 




At Each Growth Rate 






School Bv Rank 


Faculty 


1.0%, 








3,0% 


Professor 


243 


2.9 


6.2 


18.1 


"41.2 


67.5 


School 'A' 


154 


0.0 


1.9 


11.0 


37.7 


67.5 


School 'B* 


32 


0.0 


0.0 


0.0 


18.8 


50.0 


Srhnnl 'C 

UOl l\J\J\ v/ 


36 


19.4 


33.3 


63.9 


80.6 


94.4 


School 'D' 


9 


0.0 


0.0 


0.0 


11.1 


33.3 


Qrhool 'P' 


9 


0.0 


0.0 


44.4 


66.7 


77.8 


School 'F 


3 


0.0 


0.0 


0.0 


0.0 


0.0 


Associate Professor 


236 


17.8 


29.2 


50.8 


69-1 


81.4 


School 'A' 


135 


9.6 


23.0 


51.1 


74.8 


89.6 


School 'B' 


32 


0.0 


0.0 


28.1 


40.6 


56.3 


School 'C 


41 


68.3 


82.9 


90.2 


97.6 


100.0 


School 'D' 


11 


0.0 


0.0 


0.0 


18.2 


36.4 


School 'E' 


12 


8.3 


33.3 


41.7 


50.0 


50.0 


School 'P 


5 


0.0 


0.0 


0.0 


20.0 


40.0 


Assistant Professor 


180 


55.0 


62.2 


65.6 


71.1 


75.0 


School 'A' 


107 


57.9 


68.2 


71.0 


75.7 


78.5 


School 'B' 


30 


23.3 


26.7 


33.3 


46.7 


56.7 


School 'C 


30 


83.3 


83.3 


83.3 


83.3 


86.7 


School 'D' 


6 


50.0 


66.7 


66.7 


83.3 


83.3 


School 'E' 


3 


33.3 


33.3 


66.7 


66.7 


66.7 


School 'F 


4 


25.0 


25.0 


25.0 


25.0 


25.0 



Table 4 



Departments with Over Fifty Percent of 
Their Doctoral Faculty Salaries 
Falling Below the 2% Growth Rate 





Number in 


Salary Below 2% Growth Rate 


Dep^artment 


Department 


N 


Percent 


A - School 'A' 


2 


2 


100.00% 


B - School 'C 


2 


2 


100.00% 


C - School 'C 


12 


12 


100.00% 


D - School 'C 


20 


20 


100.00% 


E - School 'E' 


6 


6 


100.00% 


F - School 'C 


13 


12 


92.30% 


G - School 'C 


17 


15 


88.20% 


H - School : A' 


14 


11 


78.60% 


I - School 'A' 


27 


20 


74.10% 


J - School 'C 


27 


20 


74.10% 


K - School 'A 


21 


13 


61,90% 


L - School 'A' 


32 


19 


59.40% 


M - School 'A 


26 


14 


53.80% 


N - School 'A 


15 


8 


53.30% 




154 



182 



salaries . 

Table 4 shows the departments where at least half of the faculty 
with doctorates have salaries below those associated with a 2.0% 
annual growth rate. These fifteen departments account for 174 (61.7%) 
of the 282 faculty members with salaries below this arbitrary growth 
rate. School ' C', with six of the seven departments appearing on this 
list, is disproportionately represented here. Over 46 percent of the 
faculty with lagging salaries are in these six departments, and 
represent over 75 percent of the faculty within School ' C' . 
Snyder Method 

Following the basic premise outlined in the 1992 study on salary 
compression by Snyder, et. al . , analyses were made within each rank 
for each academic division. Salaz-y comparisons were made within each 
rank using standardized average salaries and years ' " rank. Ratios 
were calculated for predetermined time -in- rank intervals by dividing 
the average salary of the faculty in the given time interval by the 
overall average salary of the faculty of the rank being analyzed 
within given academic divisions on the Oxford campus and for all 
faculty within rank on the branch campuses irrespective of department. 
Years-in-rank intervals were adopted depending on which rank was being 
used in the analysis. For assistant professors, the time intervals 
were 0-1, 2-4, 5-7 and 8 or more, because of the seven years to tenure 
rule. For professors and associate professors the intervals were 0-5, 
6-10, 11-15 and 16 or more, due to the longevity that occurs within 
these ranks . 

Figures 2 through 4 graphically display the following results of 
these analyses where ratios of 1.0 or greater indicate that the 
average salary of the faculty within the given years -in-rank increment 
is higher than the overall average salary for faculty in that school: 

1. Full professors in Schools 'A', 'D' and ' F' and on the 
branch campuses have ratios that rise consistently as their 
years-in-rank rise (Figure 2), indicating progressively 
less compression . 

2 . For f ul 1 professors with years - in- rank of 11-15 years 
(Figure 2), their ratios are below those who have been at 
full professor for 6-10 years in Schools 'B 1 , 'C' and 'E'. 
That is, for the faculty in these schools, salary 
compression occurs after 10 or more years as a full 
professor . 

3 . Looking at associate professors who have been in rank for 
16 or more years (Figure 3), they have consistently one of 
the lowest ratios in Schools 1 C' , 'D' and 1 E' . 

4. In Schools 'A' and 'B' and on the branch campuses, the 
ratios for associate professors rise as their years in rank 
increase, thereby showing an overall lack of salary 
compression within these divisions (Figure 3 ) . 

5. At the assistant professor level (Figure 4), faculty with 
less time in rank had higher ratios in Schools 'C 1 and ' F ! . 
In School ' C' , the assistant professors most recently hired 
are paid higher on average then those who have been in the 
schools from 5 to 7 years ; for the School 1 F 1 the new hires 
are paid more than those who have been in rank from 2 to 4 
years . 

As a follow up to the basic analysis, individual ratios were 
calculated using the overall salary in rank as the divisor. These 
data are then made avai lable to the Provost , where actual salary , 
individual ratio and divisional ratio ( for comparable rank and years 
in rank faculty within same division) are given for each school and 
branch campus at each rank. Attention may want to be focused cn those 
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faculty who fall into the groups that displayed salary compression 
through :he analysis using the second methodology and also those whose 
individual ratios fall below their group mean ratio which may be 
indicative of more pronounced salary compression. Of course, 
considerations must be given to not impacting individual merit 
increase decisions . 
Conclusions 

In the first method, where individual faculty salaries are 
contrasted to a hypothetical market salary that is then used in growth 
rate analysis, faculty are not separated by rank or years in rank to 
determine the extent of salary compression. Rather they are grouped 
together, allowing only for the inclusion of years since doctorate to 
define the presence of salary compression . In the second method , 
faculty are grouped by rank and years in rank to determine where 
salary compression may be occurring at this university . Because of 
these vastly different methods, results must be interpreted 
independently of each other and the conclusions drawn by each method 
must be weighed according to the appropriateness of the method. 

For example, in both methods, faculty in School ' C appear to be 
affected most strongly by salary compression. For the other 
divisions , it was dependent on the method as to whether there was 
substantiated evidence for salary compression. 

Upon review of these two methods of determining salary 
compression, it is suggested that the first method discussed, which 
uses arbitrary growth rates and predicted salaries based on 
hypothetical new assistant professor salaries, be abandoned in future 
salary compression analyses. Perhaps a modification of the first 
method could be used which draws on actual salary increases to create 
a growth rate construct more reflective of what has actually occurred 
in the past. Although the literature on salary compression is 
extremely limited, over the next year we will attempt to discover 
other procedures for identification of salary compression beyond that 
of Snyder, et.al. (1992) and determine their appropriateness for 
implementation here. 
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DEMOGRAPHIC AND ECONOMIC OUTLOOK FOR THE 1990' S 



Selected Factors Affecting Westchester Community College and 
Higher Education in the 1990s in Westchester County and the Region 



In serving the needs of higher education in Westchester county in the 1990s, 
several of the major factors to consider are located "off campus" in the 
Westchester community at -large. They are the demographic and economic 
trends already underway in many of the towns and villages in Westchester 
county and the nearby metropolitan region. 

This report seeks to identify these factors and discuss them in terms of the 
planning that needs to occur at Westchester Community College in order to 
"be there" when the time arrives to serve effectively and adequately the 
educational needs of our perspective students. 



While the population of Westchester county (874,866 in the 1990 census) is 
expected to remain relatively constant through the year 2000, four 
significant shifts within the population are anticipated in this decade. 
They are: 



1 . The continued increase in population in the northern section of 
the county with a continued decrease in the southern region, 

2 . The growth in the number and percentage of foreign-born 
residents, particularly Hispanics and Asians. 

3. A striking change in the age structure as the county's population 
gets older. 

4. An increase in the numb*.- of women entering in the labor force. 



POPULATION TRENDS 
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(1) A SHIFT IN POPULATION TO THE NORTH 



In the last two decades a gradual increase in population has occurred in the 
northern section of the county with a concomitant decrease in the southern 
region. Between 1980 and 1990 the population in the north increased by 
19,990, while the population decreased by 15,023 in the south, a spread of 
almost 45,000 people. In the mid-section of the county the population 
remained fairly stable with an increase of 3,343. 

With regard to these shifts several qualifications must be made. The 
decline in the population in the southern section of the county is slowing 
down. From 1970 to 1980 the population there decreased by 6.1%. Between 
1980 and 1990, however, it decreased by only 3.5%. 

At the same time, however, the growth in the population in the northern 
section of the county is increasing at an increasing rate, as shown in 
Table I. Between 1970 and 1980 the population increased by 5.0%. Between 
1980 and 1990 it increased by 8.8%. In the middle section of the county, 
the growth rate is almost static with an increase of +1.6 between 1980-1990, 
having been preceded with a decrease of -4.6% in the previous decade. 



(2) AN INCREASE IN FOREIGN-BORN CITIZENS 

While the shift in the population up county would indicate that the location 
of future off-site campuses should be placed in the northern section, such 
plans must also be considered in light of the students who attend WCC and 
those whom we are seeking to attract to the college. 

In Fall 1992, 30% of our student body (full-time and part-time credit 
students) v/ere minorities. This represents almost a doubling in the 
percentage of minorities at WCC over the previous decade. Of this group, 
Hispanics and Asians comprise almost half, with Blacks comprising the other 
half. Further increases in the number of f oreign-borns , in particular, can 
be anticipated, given the aggressive admissions programs to reach high 
school students with minority backgrounds and the anticipated continued 
growth of this population in the decade ahead, especially among the Hispanic 
population . 

To bring home the point, the foreign-born population of Westchester county 
increased from 126,866 to 158,597 during the 1980s, census figures show. 
Foreign-borns now represent 18.1% of the county. According to a six page 
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TABLE I 




special feature report in the Reporter Dispatch ,* the growth of the Hispanic 
population is the factor primarily behind these large increases and is 
located primarily in southern Westchester. 



They are Mexicans in New Rochelle, Cubans 
in North Tarrytown, Ecuadorians in Ossining, 
and Peruvians in Post Chester. From 1980 to 
1990 Westchester's Hispanic population swelled 
89*2% Natives of Latin and Central America 
and the Carribbean, including Puerto Rico, 
whose residents are American citizens now 
comprise 10% of the county.* 

Moreover, another foreign-born group, those from India, increased by 143% 
mainly settling in New Rochelle, Yonkers and White Plains. 



Blacks: It should be noted that Blacks continue to constitute the largest 
minority group in the county, but both the absolute and percentage increases 
were not as great as that of other racial groups in the past decade. Blacks 
now make up 13.7% of the total county population as compared with a 1980 
percentage of 12.1%. This is a much smaller increase for this group than 
had been projected for Westchester. 

In short, with the exception of two municipalities in the northern section 
of the county, Peekskill and Ossining, most of the foreign-born and minority 
populations are currently located in southern Westchester: Yonkers, Mount 
Vernon, and New Rochelle and Port Chester. To the extent WCC seeks to serve 
this group, decisions must be made in light of the fact that this section of 
the student body and its future population growth are likely to remain in 
the southern section of the county. 

Racial Makeup by Age: Table II adds further pertinent information to 
college decision making in that it shows each race broken down by major age 
brackets. Hispanics (24.5%) and Asian (26.2%) had the largest percentage of 
people in the 0 to 14 Yrs age bracket (read future perspective WCC student) , 
while whites (16.9%) and Blacks (22.9%) had the lowest. Planning that looks 
down the road more than five years, must take this into consideration. 



* "A New Wave of Arrivals," Reporter Dispatc h, Section E; Sunday, July 4, 
1993, pp. 1-6. 
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(3) AGING OF THE POPULATION 



In this decade the population will show a decided shift in age structure 
from young to older citizens. The median age in the county was 3 6.2 in 
1990, an increase over the 34.5 reported in 1980 and an indicator of the 
continuing impact of the growing population 65 and over. 

The under- 3 5 age group is expected to shrink as more of the population 
shifts to the over-35 age category. The largest increase will be seen in 
the 40 to 55 year old cohort. The increase in the number of seniors will 
also be significant. This age group increased by 10.5% (11,867) in the past 
ten years and in 1990 represented 14.4% of the total population. 

With the aging of the population will also come an increase in the 
percentage of women who comprise it. According to the 1990 census, the 
number of women exceeded the number of men by 44,306 or 5.1%. As the 
population continues to age, this percentage is expected to increase. 

Baby Boom Echo: Despite the "aging" of the population, the greatest gain 
between censuses was registered by the under 5 (pre-school) population which 
showed a 22.7 % increase over the total from 1980. This is due to a 
steadily rising birth rate in the county since 1983, a phenomenon sometimes 
called the "Baby Boom Echo." Around 1997-1998, therefore, an increase in 
high school graduates in Westchester will be occurring. 



TABLE II: WESTCHESTER POPULATION BREAKDOWN BY AGE AND RACE-1990 
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(4) AN INCREASE OF WOMEN IN THE WORKFORCE 

Much of the recent growth in the last decade in Westchester's labor force 
has been due to dramatic increases in the number of women joining the labor 
force. In preparation for their joining the labor force, many women 
furthered their education at WCC. Since 1970, the percentage of women 
attending WCC full-time has gone from 31.7% to 49.7%. The percentage of 
women attending part-time has sky rocketed even more dramatically from 33.7% 
to 60.9%. 

Demographic and labor force participation trends indicate that in the decade 
of the 1990s women will continue to join the labor force in growing 
numbers.* According to New York State Labor Department projections, the 
female labor force participation rate in the Hudson Valley Region is 
expected in increase from just under 56 percent in 1987 to slightly over 68 
percent in the year 2000. As a result, the female share of the total labor 
force is projected to expand from 44 to 50 percent over the period making 
the percentage of females comprising the labor force equal to males. 

Assuming women continue to return to school before joining the labor force, 
thsy will remain a significant pool of new student applicants for WCC. 
S kills and courses leading to professions particularly attractive to women 
living in the Westchester region should be in in c reased demand at WCC. 

The New York Labor Department cautions that its projections assume, 
"barriers which hinder women from entering the labor force, such as a l.*ck 
of child care and inflexible work schedules, will be removed."* By the same 
token, WCC must continue to remove these barriers, too. 

Minority Group Members: At the same time white, non-Hispanic men will make 
up a smaller share of the labor force and minority group members will make 
up a larger share. The 1990 census showed that minority groups increased in 
the Hudson Valley Region by 43% (120,700) while whites contracted by 2% 
(26, 000) .** 



Persons With Disabilities or Handicaps: Persons with disabilities will also 
increase considerably in the workforce. According to the 1990 census, 
approximately 81,600 residents 16-64 years of age in the seven- county Hudson 
Valley Region have a work disability.*** These individuals represent an 
large pool of potentially qualified job applicants for employers willing to 
utilize then. The ADA, which was signed into law July 26, 1990, and took 
affect in 1992, bars discrimination in employment and requires most 
employers to make reasonable accommodations for qualified employees with 
disabilities . 



* "Tomorrow's Jobs, Tomorrow's Workers, Hudson Valley Region, 1992;" New 

York State Department of Labor, Division of Research and Statistics, 

Binghamton, New York, page 3. 

** Ibid. , page 4. 

*** Op.cet . , pp. 5 & 6. 
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ECONOMIC OUTLOOK 



A prime mission of Westchester Community College is to train the workforce 
of the county, both in basic academic skills, such as mathematics, reading, 
and writing, and in specific trades and professions such as nursing and 
accounting. 

Corporate restructuring, cuts in Pentagon procurement, and government belt- 
tightening are producing a fundamental transformation in the Hudson Valley 
economic region which have already had serious implications for the economy 
and future job opportunities. They promise, also, to have serious 
implications for Westchester Community College, particularly in the types of 
courses that future WCC students will be demanding. 

Updating the curriculum to meet the changed business environment of the 
1990s in Westchester county will be essential if WCC is to keep attracting 
and serving the needs of students seeking higher education. 



THE LONG-TERM ECONOMY 

Westchester Community College is fortunate to be located in a county with a 
relatively favorable long-term economic outlook. According to the Office of 
Commerce and Economic Development of Westchester County, the county is 
poised for growth, although at a much lower rate than in the previous 
decade.* The number of jobs grew 15 percent during the 1980's, but the 
pattern of growth was not consistently positive. Employment did not grow at 
all in the late 1980s. 

Because the 1990-1992 recession has taken its toll on jobs in the region, 
and because the Westchester economy is structurally similar and frequently 
interdependent with New York City's, county and regional economists 
anticipate that it will take some time before Westchester reaches previous 
employment levels and resumes a sustained economic growth. When it does, 
the landscape will have changed in terms of where employment opportunities 
exist for future WCC graduates. 

Shift to Service Economy: The most pronounced change in Westchester's 
economy is a shift to the service economy accompanied by a decline in 
manufacturing activity, as shown in Table IV. Continuing a trend that 
began in the 1980's, future job generation will be concentrated in service- 
producing industries. From 1986 to 1991, employment in industries that 
provide services expanded by more than 33,000 or 5 percent in the Hudson 
Valley Region. 



* "Westchester Economy, Long-Term Highlights, " a paper by Noreen Preston, 
Economic Development Specialist, Westchester County Office of Commerce and 
Economic Development, 1992. 
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Many of these services are labor intensive and require considerable 
training. Expansion in the service sector has been fueled, in particular, 
by a growth in health care facilities, and the need for services supplied by 
small and medium-sized businesses and law firms as larger companies down- 
size and use external services. 

TABLE III: DISTRIBUTION OF JOBS BY INDUSTRY IN THE HUDSON VALLEY 



The goods producing sector , comprised of construction and manufacturing 
industries, will continue to decline. Manufacturing payrolls, contracted 
25,000 or 17 percent from 1986 to 1991. As a result, manufacturing now 
accounts for only a small share of Westchester's jobs, about 13 percent. 
One of the few industries showing some resiliency in this sector, however, 
is printing and publishing. 

The finance, insurance and real estate industries , which have been growing 
rapidly in Westchester, have been facing major constraints in the 1990-92 
recession. The long-tex*m favors continued growth, but the next few years 
are likely to erratic* 

The government sector has maintained a relatively constant share of the 
county's jobs, approximately 15%, Government job increases have occurred, 
slowly and steadily at the state level within the county, but local 
government predominates with many workers employed with school systems or 
other educational institutions. 

The unemployment rate in Westchester is expected to be below most market 
areas in the metropolitan region. The long-term range should be between 
3 1/2 percent and 4 1/2 percent. 
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THE JOB MARKET 



According to the the New York State Department of Labor, an estimated 30 , 230 
jobs are expected to be created from 1992 to 1996 in the Hudson Valley- 
Region.* This should have a positive impact on the College and student 
enrollments . 

The two main growth industries projected for the Hudson Valley region 
between 1991 and 1997, are Health Services and Business Services, as shown 
in Table IV. Jobs in the Professional Services area and Social Services 
area are also projected to grow, but not at the same pace as health services 
and business services,* 

TABLE IV: PROJECTED GROWTH INDUSTRIES IN THE HUDSON VALLEY REGION* 



* "Tomorrow's Jobs, Tomorrow's Workers, 1992 Hudson Valley," New York State 
Department of Labor, Division of Research and Statistics, p, 11. 
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The two specific occupations expected to have the largest growth are home 
health aids with a projected increase of 25.6% or 2,100 people, and systems 
ana lysts with a projected increase of 25.3% or 800 people, as shown in Table 
VI. In both cases, these occupations appear to be well suited for WCC in 
terms of providing training and higher educational needs. 

Not far behind these two specific occupations in terms of growth projections 
are medical assistants, medical secretaries, marketing, advertising, public 
relations specialists, accountants and auditors, and receptionists - 
information clerks. Table VI shows the anticipated percentage increase and 
net increase over the 1991-1997 period. 



TABLE VI: PROJECTED LARGEST GROWTH OCCUPATIONS 
Hudson Region Valley 1991-1997 





Occupation 


Percent 
Chanqe 


Net 
Chanqe 


1. 


Home Health Aids 


25.6% 


2, 100 


2 . 


Systems Analysts 


25.3% 


800 


3 . 


Medical Assistants 


21.4% 


400 


4 . 


Medical Secretaries 


20.5% 


500 


5. 


Marketing, Advertising, 
Pub. Relations Mgrs . 


15.7% 


500 


6 . 


Accountants and Auditors 


13 .0% 


900 


7 . 


Receptionists , Informa- 
tion Clerks 


13 .0% 


GOO 
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HIGH SCHOOL GRADUATES 



As the percentage of students in the over- 21 age brackets increase at WCC, 
the importance of the size of the high school graduate pool in Westchester, 
decreases. Nevertheless, this pool of applicants is still very important. 
In Spring 1993, 34% of the full-time students were in the 18-19 years 
category and another 26% were in the 20-21 category for a total of 60% of 
the full-time student body. The percentage of part-time students under 22 
is far less (19%), but still significant, 

In the last two decades, the number of high school graduates from 
Westchester schools has decreased from 14,389 in 1975 to 8,297 in 1991, as 
shown in Table VII. Were it not for the existence of two important factors, 
these figures would look grim, indeed. 

The first factor is that beginning around 1997 the "Baby *oom Echo" will 
take effect, resulting in an increase in the number of students graduating 
from Westchester high schools. According to the U.S. Census, since 1983 
there has been a steadily rising birth rate in the county, attributable to 
the fact that the baby boomers of the 1950s are of child bearing age. 

The second factor of note is the very laudable track record that WCC has had 
since 1975 in steadily increasing the percentages of Westchester high school 
graduates who elect to attend WCC. The percentage has gone from 9.17% in 
1975 to 15.9% in 1991, the latest year with available statistics. 

If WCC can hold its "market share" or even increase it slightly during this 
decade, student enrollments will be enhanced. The projected larger than 
average growth in Hispanics in the college age bracket would make this group 
a particularly important one to cultivate. 

TABLE VII: PERCENT OF WESTCHESTER HIGH SCHOOL GRADUATES ATTENDING WCC 
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COMMUTATION 



Transportation improvements in Westchester County and the Hudson Valley 
Region continue to shorten the commutation time to Westchester Community 
College from various points throughout the metropolitan region. 

Twelve major New York State Department of Transportation road projects are 
either underway or have been completed recently in Westchester as a part of 
the 1988 Rebuild New York Bond Act. Improvements on the Taconic State 
Parkway at the Hawthorne Interchange to Pleasantville Road, the Cross 
Westchester Expressway from from the Tappan Zee Bridge to Harrison and the 
Bear Mountain Bridge Road in Cortlandtd, in particular, will cut commuting 
time from points in the northern section of the county and beyond. 

Other traffic management programs and organizations have become an integral 
part of the road improvement process in the county. Metropool offers 
commuter pool matching, distributes the Commuters 1 Register and sponsors van 
pools. Metro Traffic Control, the nation's largest traffic reporting 
service, has extended service to Westchester on a 24 -hour, 7~day-a week 
basis. The Shadow Trafffic Network traffic information service covers road 
conditions, mass transit, congestion, hazardous areas, length of expected 
delays, and accidents for a 75-mile radius of the New York Metropolitan 
area. The Westchester County Samaritan currently sponsored by Texaco, 
patrols 1-287 to aid disabled vehicles and assist in emergency situations. 

In light of the major road improvements and increased traffic support 
systems which are reducing the commuting time, the geographical area served 
by WCC has been and should continue to be expanding in this decade. This is 
especially so for the Putnam and Dutchess county region. 

Future plans for the College should include consideration of the special 
needs of students commuting long distances including: 



1. 



3 . 

4 . 



5 . 



Car pool matching 

Two- days -a -week programs 

Job market analysis of Putnam and Dutchess counties 
The creation of a better home-away- from-home 
atmosphere . 

College representation on Metropool and other traffic 
systems organizations . 
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EXECUTIVE SUMMARY 



For purposes of strategic planning, several of the most important factors to 
consider are located "off campus" in the Westchester community at-large. 
This report identifies four factors, (1) Population Trends, (2) Economic and 
Job Market Trends, (3) High School Graduates, and (4) Commutation Trends, 
and discusses them in terms of their impacu on the College in the future, 
particularly as it relates to the composition of the student body, and the 
location of off -site campuses. 

Populations Trends: With regard to population trends in Westchester, four 
majors shifts within the county are projected to occur or already are 
occurring in this decade: 



1. A continued shift of the population from the southern 
section of the county to the northern section. 

2. A growth in the percentage of foreign-born residents, 
particularly Hispanics and Asians. 

3. A striking change in the age structure of the county's 
population as it ages. 

4 . A continued increase in the number of women entering the 
labor force. 



While the population is shifting north, growth in the Hispanic, Asian and 
Black populations continue to be centered in the southern cities of the 
county. Decisions in locating off -site campus, should take this into 
consideration. 

Economic and Job Market Outlook: With regard to the economic outlook of 
Westchester, the college is fortunate to be situated in a county with 
relatively favorable, long-term growth prospects, although economists agree 
that the 1990-1992 recession will take time to come out of and the rate of 
growth thereafter will be much lower than in the previous decade. 

According to the New York State Department of Labor, 3 0,230 jobs are 
expected to be created from 1992 to 1996 in the Hudson Valley Region. 
The two big growth industries will be in Health Services and Business 
Services. The specific occupations projected to grow the fastest are: 



1 
2 
3 
4 
5 
6 
7 



Home Health Aids (25.6%) 
Systems Analysts (25.3%) 
Medical Assistants (21.4%) 
Medical Secretaries (20.5%) 

Marketing, Advertising, Public Relations (15 . 7%) 
Accounting and Auditors (13.0%) 
Receptionists, information Clerks (13.0%) 
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High School Graduate Pool : While the college has diversified in the age of 
its students, the pool of high school graduates is still very important. A 
bright spot is on the horizon, here. Around 1997 the "Baby Boom Echo" will 
take effect as the baby boomers' children reach graduation age. 

According to the U.S. Census, since 1983 there has been a steadily rising 
birth rate in the county, attributable to the fact that the baby boomers of 
the 1950s are of child bearing age. If WCC can continue its laudable track 
record of steadily increasing the percentage of Westchester high school 
graduates who attend the college (the percentage has gone from 9.17% in 1975 
to 15.9% in 1991) or even remain constant, the increased high school 
graduate pool should result in increased enrollments. 

Commutation Time: Transportation improvements in the Westchester county and 
Hudson Valley region continue to shorten the commutation time to Westchester 
Community College from various point throughout the metropolitan region. In 
light of this, the geographical area served by WCC should continue to expand 
in this decade. 

Future plans for the College should include consideration of the special 
needs of students commuting long distances such as (1) car pool matching, 
(2) two -days -a- week programs, (3) job market analysis of Putnam and Dutchess 
counties, and (4) college representation on Metropool and other traffic 
systems organizations. 
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The State University at Buffalo has two campuses three miles apart, and is home to over 
25,000 students, of whom more than 17,000 are undergraduates. Campus lore and anecdotal 
evidence indicates that a major problem students have with the university (parking and 
drop/add aside) is the feeling that they are H simply numbers," not individuals with individual 
backgrounds and expectations. Although this is not an uncommon perception among students 
at large institutions, the university is logically concerned that the college experience be a positive 
one during which students develop as individuals and members of a social organization. For 
that reason, an assessment of initial attitudes and expectations-particularly as they relate to 
changes during the college experience and the likelihood of students persisting to graduation 
-is an important part of the on-going research with the Office of Institutional Analysis. 

Results of the Fall 1985 Annual Freshman Survey, given during summer orientation to 
freshmen entering the University at Buffalo for the fall semester, were analyzed with respect 
to the probability of students persisting at the university and obtaining a degree within five 
years. Items to be analyzed were selected on the basis of the potential of the university 
experience to impact on them in some way. Primary emphasis was placed on the following 
variables: students' concern about financing their college education; self-ratings on intellectual 
and social skills; reasons for going to college and expectations of the college experience; and 
further educational plans. 

Concern over Financing Their Education. Students were asked to rate their concern about their 
ability to finance their college education. Generally, students with some or a major concern 
about finances were less likely to graduate within the time period included in the present study. 
Possible interpretations include (1) failure to graduate within the time period is directly related 
to financial concerns-e.g., because a student had to work part or full time and could not take 
a full course load, (2) students leave the university because of an inability to meet tuition or 
other expenses, or (3) worry over finances made it difficult to concentrate on studies. 

Reasons for Going to College. Students who cited finding a better job as a very important 
reason for going to college were more likely to obtain a degree; the same was true for students 
who said that making more money was a very important reason for going to college. Less 
"pragmatic" reasons such as developing a philosophy of life or learning more about things that 
interested them did not appear to differentiate well between those students who attained 
degrees and those who did not. 

Expectations of the College Experience. A larger proportion of students who indicated that 
there was a very good chance that they would change majors graduated within five years than 
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was true of students who indicated that there was very little chance that they would change 
majors. It appears that expecting to be satisfied with college is correlated with the likelihood 
of obtaining a degree. Nearly 60% of the students who said there was a very good chance they 
would be satisfied had graduated within five years; about 56% of those students who felt there 
was some chance they would be satisfied, and about 45% of those students who believed there 
was very little chance of being satisfied, had graduated within the time period analyzed. 

Hi ghest Degree Planned. Students were asked about the highest degree they planned to obtain, 
and what they expected to be the highest degree earned here at the university. No definitive 
pattern emerged with respect to graduation rates. Analysis was then limited to the two 
categories containing the most respondents: those students expecting to earn bachelor's or 
master's degrees either here or elsewhere. Little difference was found at the bachelor level. 
However, students planning to obtain master's degrees at the University at Buffalo obtained a 
bachelor's degree at a slightly higher rate than students planning to earn a master's degree at 
another institution. 

Self-Ratings. In general, students who rated themselves above average or in the highest ten 
percent in a variety of categories, compared to their peers, graduated at higher rates than those 
students who rated themselves near the bottom. We found the results of the analysis of 
students self-ratings to be most interesting, and so present them in greater detail. 

Our research cohort was the group of first-time, full-time freshmen entering the university 
in the fall semester of 1985. Students were asked to rate themselves, relative to their peers, on 
a number of factors. These included: academic ability, artistic ability, drive to achieve, 
emotional health, leadership ability, mathematical ability, physical health, popularity, intellectual 
self-confidence, and social self-confidence. Response options were "Highest 10 %", "Above 
Average", "Average", "Below Average", and "Lowest 10%." Confining ourselves to students who 
remained in good academic standing, we then analyzed the results of these self-ratings and the 
probability that a student would persist at the university and obtain a degree within five years. 

There was no significant relationship between many of the self-rating categories and 
graduation. We did, however, find a statistically significant, positive relationship between 
graduation and five of the self-ratings. 

Most of the students (nearly 60%) responding to the survey rated themselves as above 
average in academic ability. About a fifth (21%) rated themselves in the highest 10%. We 
found a significant relationship between students' perceptions of their academic ability and the 
probability that they would obtain a degree within five years. The relationship was linear: 65% 
of the students who rated themselves in the highest 10%, 56% of the students who rated 
themselves above average, and 49% of the students who rated themselves average obtained a 
degree. (Only three students rated themselves below average and none obtained a degree; no 
students responding to the survey rated themselves in the lowest 10% on academic ability.) 

When asked specifically about mathematical ability, most students were a little harsher in 
their judgment. About the same proportion rated themselves in the top 10% (22%). A smaller 
proportion (40%) rated themselves above average, and larger proportions rated themselves 
average, below average, or in the lowest 10%. The proportions within each self-rating category 
were similar for the top two: 64% of the students who rated themselves in the top 10%, and 
56% of students who rated themselves above average, obtained a degree v/ithin five years. For 
students who rated themselves average, below average, or in the lowest 10% the percentages 
with degrees were 55%, 48%, and 32%, respectively. 

Nearly half of the respondents (48%) rated themselves above average in their drive to 
achieve, and nearly another fifth (19%) rated themselves in the top 10%. The relationship 
between self-rating and degree achievement was again linear: 62% of students in the highest 
10%, 58% of students above average, 53% of students who rated themselves as average, and 
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35% of students below average had obtained a degree within five years. (Only three students 
rated themselves in the lowest 10% on drive to achieve, and one had obtained a degree.) 

An interesting category related to degree attainment was emotional health. Nearly two fifths 
(39%) of the respondents considered themselves above average with respect to emotional 
health, and another two fifths (39% again) considered themselves average. Of these students, 
62% and 54%, respectively, obtained degrees.Small proportions of the respondents rated 
themselves below average or in the lowest 10% with respect to emotional health relative to their 
peers (3% and 1%, respectively); of these, 35% and 36%, respectively, earned degrees. 

The final category in which a significant relationship was found between self-ratings and 
degree attainment was social self-confidence. (Interestingly, intellectual self-confidence was not 
found to be significantly related.) The largest proportion (40%) of students rated themselves 
average; of these, 59% had earned a degree within five years. Of the 36% of the students who 
considered themselves above average in social self-confidence, 59% also earned degrees. Just 
over a third (35%) of the students in the lowest 10% obtained degrees. It is interesting to note 
that students rating themselves below average (11% of the respondents), in followed the groups 
rating themselves average and above average with respect to the proportion which obtained 
degrees (52%). Students who rated themselves in the highest 10% fared worse; 46% had 
obtained degrees within the five-year period assessed. 



Table 1. Self-rating on academic ability by degree status. 

Rating Category De gree Awarded No Deg ree 





Number 


Percent 


Number 


Percent 


Lowest 10% 


0 


0% 


0 


0% 


Below Average 


0 


0% 


3 


100% 


Average 


178 


49% 


186 


51% 


Above Average 


643 


56% 


507 


44% 


Highest 10% 


264 


65% 


142 


35% 


Total 


1085 


56% 


838 


44% 



Table 2, Self-rating on mathematical ability by degree status. 
Rating Category Degree Awarded No Degree 



Lowest 10% 
Below Average 
Average 
Above Average 
Highest 10% 
Total 



Number 


Percent 


Number 


Percent 


9 


32% 


19 


68% 


68 


48% 


74 


52% 


303 


55% 


249 


45% 


431 


56% 


345 


44% 


272 


64% 


152 


36% 


1083 


56% 


839 


44% 
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Table 3, Self-rating on drive to achieve bv deeree status. 



Ratin g Category Degree Awarded No Degree 



Lowest 10% 
Below Average 
Average 
Above Average 
Highest 10% 
Total 



Number 


Percent 


Number 


Percent 


1 


33% 


2 


67% 


19 


35% 


35 


65% 


307 


53% 


274 


47% 


532 


58% 


392 


42% 


223 


62% 


134 


38% 


1082 


56% 


837 


44% 



Table 4, Self-rating on emotional health bv degree status. 

Ratin g Category Degree Awarded No Degree 



Lowest 10% 
Below Average 
Average 
Above Average 
Highest 10% 
Total 



Number 


Percent 


Number 


Percent 


5 


36% 


9 


64% 


21 


35% 


39 


65% 


400 


54% 


336 


46% 


465 


62% 


285 


48% 


186 


53% 


163 


47% 


1077 


56% 


832 


44% 



Table 5, Self-rating on social self-confidence bv degree status. 

Degree Awarded No Degree 



Lowest 10% 
Below Average 
Average 
Above Average 
Highest 10% 
Total 



Number 


Percent 


Number 


Percent 


7 


35% 


13 


65% 


109 


52% 


100 


48% 


457 


59% 


315 


41% 


402 


59% 


279 


41% 


108 


46% 


127 


64% 


1083 


57% 


834 


43% 



We were able to measure students self-ratings on these variables as they entered the 
University at Buffalo; we were also able to assess change in those ratings after four years at the 
institution. In the spring of 1989 a follow-up survey was sent to students who had participated 
in the initial survey during orientation prior to the Fall 1985 semester. The response was not 
what we had hoped-we had a return rate of just over 22%-due in large part to the fact that 
it was a mail survey addressed to what was, in many cases, a home address at which the student 
was unlikely to receive mail during the semester. Although we did not feel confident enough 
in the representativeness of the results to attempt further analysis (e.g., correlating change in 
self-ratings with graduation), we found that the changes recorded offered data on which to base 
speculation and possibly further study. 
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We found, for example, that there was negligible change as the result of four years on the 
university campus in several of the categories. Among these variables were artistic ability, drive 
to achieve, and emotional health. In other categories-academic ability and popularity-there 
was a small change, (The proportion of students rating themselves above average or in the 
highest 10% decreased between 1985 and 1989.) Students self-rating of their intellectual and 
social self-confidence both increased slightly, as did their assessment of their writing ability. The 
largest changes were in ratings of physical health, which decreased by more than seven 
percentage points; mathematical ability, which decreased by more than ten percentage points; 
and leadership ability, which increased by nearly eleven percentage points. Of these last three 
variables, only self-ratings of mathematical ability was originally found to be correlated with 
graduation rates. 



Table 6. Proportion of students who rated themselves above average or in the highest 10%. 





1985 


1989 


Academic ability 


84% 


82% 


Artistic ability 


31% 


31% 


Drive to achieve 


70% 


69% 


Emotional health 


60% 


60% 


Leadership ability 


50% 


61% 


Mathematical ability 


67% 


57% 


Physical health 


63% 


56% 


Popularity 


41% 


42% 


Self-confidence (intellectual) 


62% 


67% 


Self-confidence (social) 


43% 


47% 


Writing Ability 


54% 


59% 
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Technical Background 



The State University of New York at Buffalo's Office of Institutional Analysis (OIA) utilizes 
a networked micro-computer processing suite to perform its decision support function for 
university management. In their most basic form, the Annual Freshman Survey data are 
presented in a format favoring the analyst possessing the skills to utilize traditional mainframe 
processing resources. In addition, the data's multiple card-image per record format makes it 
difficult to merge with existing data base (DB) tables. (Typical micro-computer DB 
environments do not efficiently process such records). Integration of the Annual Freshman 
Survey data within the OIA's processing suite presented us with an opportunity to enrich our 
pool of source-data used to measure student behavior as well as a means to challenge our 
technical expertise. The methodology used was as follows: 

1. Capture the raw data. The Annual Freshman Survey data for 1985 resided on 9-track 
magnetic tape media. (Starting in 1989, data was placed on standard 3.5" micro-computer 
diskette media). An IBM 3090 mainframe was employed to read the tape from its native 
EBCDIC format and place the results on temporary disk storage, still within the mainframe 
environment. Two files existed on the tape: the Annual Freshman Survey 1985 data file and 
the Annual Freshman Survey 1985 SPSS command file (SPSS is a very powerful statistical 
analysis package found on many different processing platforms. The inclusion of a command 
file on the tape obviated the need for the researcher to define variables within the data file; 
this was done by the command file). Both files were read from the tape. Upon completion of 
read operations, both files were captured for usage in OIA's micro-computer environment 
utilizing ethernet linkages and FTP. (Conversion of the character set from EBCDIC to ASCII 
was performed automatically via FTP). At the end of this session, two files resided on OIA's 
networked server's drive: AFS1985.DAT and AFS1985.SPS (containing the raw data and SPSS 
variable definition commands respectively). 

2. Load data into a temporary data base table. For this application, OIA utilized RBase as the 
primary DB vehicle to perform much of the Annual Freshman Survey data capture operation. 
As indicated, the Annual Freshman Survey data file's multiple card-image per record format 
(see Figure 1) presented a problem if it was to be linked to existing OIA DB tables. This is 
because, in a multiple card-image file, many rows of data make up one record. (In the case of 
the Annual Freshman Survey data file, five such images comprise one student's record). Each 
image contains a field denoting the image number (the Annual Freshman Survey SPSS 
command file specifies this as position 80). In a traditional SPSS file, each image may be read 
through a relatively simple command statement. However, within a micro-computer DB 
application, there is no easy way to perform this operation. OIA chose to define a temporary 
DB table corresponding to the basic 80 character image shown in Figure 1. The table, shown 
in Figure 2, contains enough positional definitions to identify each character of the image. As 
can be noted, the last DB field contains the image number captured from the file (field name 
"CARDNUM"). Also, fields six through sixty-five have been given "dummy" names, since this 
is the only means to provide a designation to each differing field occupying the same physical 
space within the card images. Since the data resides in a "fixed" format (each element always 
occupies the same place within an image), it is a relatively simple matter to specify where to 
capture the data from the file to place within each DB field. In execution, OIA utilized the 
RBase "Gateway" function, specified the field positions manually, and loaded the data into the 
temporary DB (AFS1985T). (Inspection of the data within this table would have produced an 
output similar to Figure 1). 
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3. Process data within the temporary table. At this stage, the Annual Freshman Survey data 
resided within the RBase DB environment. However, it was not in a form that could be easily 
utilized. What was desired was a method to reduce each record's multiple card-images down 
to one. In this manner, each student's Annual Freshman Survey responses could easily be 
linked to existing data collated from other sources (utilizing SSN as a key). To perform this 
operation, a second DB table was created. This table (AFS1985) contained all elements defined 
by the SPSS command file. To place data from AFS1985T within AFS1985, program code was 
generated to read each student's images from AFS1985T, integrate them into one, and place 
the result in AFS1985. Upon completion, all students had a single Annual Freshman Survey 
record containing 290 variables. 

4. Merge with other data. OIA has developed a massive DB containing data relating to student 
enrollment patterns over time. This is termed the "Student Flow Model" (SFM). SFM, 
although something of a misnomer since it is not a model so much as a framework for storing 
entering student information, contains both static descriptive (ethnicity, gender etc.), and 
variable semester (class hours taken, QPA etc.) data relating to a given student for as many 
semesters as the student takes classes within the institution. In practice, OIA can generate 
enrollment patterns from SFM for just about any "cohort" (ex. asian transfer students from two- 
year public institutions who enter the University's engineering program). Also, OIA can capture 
mainframe-resident on-line data relating to a variety of study options via the ethernet network 
referenced in #1. We had determined earlier that inclusion of Annual Freshman Survey data 
within the SFM environment was a highly desirable course of action. Since the SFM contains 
over 140 student-related variables, merging these tables yields a pool of over 400 variables with 
which the researcher can analyze many aspects of student behavior. RBase, while not the most 
efficient DB package in terms of "horsepower", does perform such merges fairly easily. 
Accordingly, the Annual Freshman Survey data was merged with SFM data, and, in this case, 
graduation information not contained within the SFM to generate the final DB structure, 
AFS1985F. 

5. Extract specific fields for analysis. Not all 400-plus items from the final merged table were 
required for the analysis outlined within this paper. In addition, the primary analyst conducting 
the study was more comfortable working with data in an xBase (FoxBase, DBase) DB 
environment. To facilitate her analysis, selected fields from within the merged file were 
converted and ported to an xBase DB format (AFS1985.DBF). The formal analysis of the data 
was then undertaken using FoxBase version 2.1 and SPSSPC version 4.1. 
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Figure 1: Annual Freshman Survey Individual Record Card-Image 



R ECORD #1 



IMAGE NUMBER 



18880102 0925876281031124175111111111105002045431111216636321111111112211211211 
18880102 0925876281111222111111131111213111231321132122222425555455554333133332 
18880102 0925876282321301420647473432111122433141443332400000000000000000065143 
18880 102 09258762845622647848214 1500144223443221422234441444444444 1332343 1 1 1444 
18880102 092587628332112123554211111111111111111211111111111111111111221 5 



R ECORD #2 

18880104 1074412941031125154112111211100053344657144110000000000000002222222111 
18880104 1074412941111232111111111311211211111213112221122423414344445321123332 
18880 104 10744 1294 1 1 123 1 166060648334423 1 13 124344 1 1 1 13 143 1 1321 121 1 1 1 133 122266443 
18880104 1074412944511461513311322001231143321112111111322141413131111141111444 
18880104 107441294132112223512412111111111111111111111111111111111H2121 5 



* 



RECORD #N 

99999999 9999999999999999999999999999999999999999999999999999999999999999999991 
99999999 9999999999999999999999999999999999999999999999999999999999999999999992 
99999999 9999999999999999999999999999999999999999999999999999999999999999999993 
99999999 9999999999999999999999999999999999999999999999999999999999999999999994 
99999999 999999999999999999999999999999999999999999999999999999999999999 5 
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Figure 2: Temporary DB Structure to Hold Annual Freshman Survey Ca rd-Image 



FIELDNAME TYPE LENGTH 



1 ACE TEXT4 characters 

2 SHRED TEXT2 characters 

3 GRPA TEXT2 characters 

4 GRPB TEXT2 characters 

5 SSN TEXT9 characters yes 

6 Fl TEXT1 characters 

7 F2 INTEGER 

8 F3 INTEGER 

9 F4 INTEGER 

10 F5 INTEGER 
11 F6 INTEGER 

12 F7 INTEGER 

13 F8 INTEGER 

14 F9 INTEGER 

15 F10 INTEGER 

16 Fll INTEGER 

17 F12 INTEGER 



* 



59 F54 INTEGER 

60 F55 INTEGER 

61 F56 INTEGER 

62 F57 INTEGER 

63 F58 INTEGER 

64 F59 INTEGER 

65 F60 INTEGER 

66 CARDNUM INTEGER IMAGE NUMBER 
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I 

I 

ANNUAL FRESHMAN SURVEY SPSS COMMAND FILE _ 



FILE HANDLE CIRP1986 

DATA LIST FELE=CERP1986 RECORDS =5/ 

ACE1 1-4 SHRED 1 5-6 GRP11,GRP21 7-10 SSN1 11-19 

SEX 20 AGE 21-22 TWINSTAT,YRGRADHS,STATUS,NDEPPAR,NDEPCOL,HSGPA, 
HSRANK,MARITALPREVCRED,CRED 1 TO CRED4,NOCRED 1 TO NOCRED4 23-39 
HTDEGALLHIDEGHRE 40-43 PLANXJVE,PP^FLTVE ) CHOICE,DISTHOME,NUMAPP ) 
NUMACQAEDOl TO AID2LDEPQ851 TO DEPQ853,DEPQ861 TO DEPQ863, 
RACE1 TO RACE3 44-79 CARDED 1 80/ 
ACE2 1-4 SHRED2 5-6 GRP12,GRP22 7-10 SSN2 11-19 

RACE4 TO RACE7,CrnZEN,PARSTAT,DISABl TO DISAB8,ACT8501 TO ACT8526, 

SLFRAT01 TO SLFRAT12,REASON01 TO REASON08 20-79 CARDED2 80/ 

ACE3 1-4 SHRED3 5-6 GRP13,GRP23 7-10 SSN3 11-19 

REASON09 TO REASONL,FTNCONJPOLTVIEW 20-24 INCOME 25-26 

FATHED,MOTHED 27-28 SCAREER,FCAREER,MCAREER 29-34 

VTEWS01 TO VIEWS23,CHOOSE01 TO CHOOSE15,SRELIG,FRELIG,MRELIG, 

HSSTUDY1 TO HSSTUDY4 35-79 CARDED3 80/ 

ACE4 1-4 SHRED4 5-6 GRP14.GRP24 7-10 SSN4 11-19 

HSSTUDY5 TO HSSTUDY8,HRSPWK01 TO HRSPWK12 20-35 MAJOR 36-37 

GOALS01 TO GOALS18,FUTACT01 TO FUTACT24 38-79 CARDED4 80/ 

ACE5 1-4 SHRED5 5-6 GRP15.GRP25 7-10 SSN5 11-19 

FUTACT25 TO FUTACT26,PERMIT ) OPTQ01 TO OPTQ10,FCAR01 TO FCAR18, 
MCAR01 TO MCAR21,RESPRACE,NORMSTAT 20-73 CARDED5 80 
COMMENT 



(Additional lines follow but are omited for illustration purposes) 
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INTEGRATED ANNUAL FRESHMAN SURVEY TABLE 



T7TT7T T> XT A \vTTh 


TVPF 


1 . \ \ 1 lVl XXX 


1 ACJb 


TpYT 


4 characters 




TFYT 

1 xj/V X 


9 characters 


1 PT5DA 

J oKJr A 


TpYT 

xx^v X 


*7 characters 






9 characters 


c ccvr 


TPYT 

XXwV 1 


Q characters ves 


A QPY 
0 oxlA 


TPYT 

XXwV X 


1 characters 




TPYT 


*? characters 


o IWllNoxAl 


TPYT 


1 characters 


y I xxolsAxJxxo 


TPYT 
X JD/VX 


1 characters 


1H T7TTT T QTAT 


TPYT 

X JC//V X 


1 characters 


1 1 WTYPPP A 1? 


TPYT 


1 characters 




TPYT 


1 characters 


11 TJTCr^P A 
U noLrr A 


TPYT 
X X 


1 characters 


i/i ircn? a xrv 
14 riot^xvATNXV 


TPYT 


1 characters 


1^ Y/T At? rT A T 
XJ IVlArvll/VLr 


TPYT 


1 characters 


10 x x\Jti V v^x\JZ/U 


TPYT 

X Xw/V X 


1 characters 

X VXlvUL ttv Ivl O 


1*7 rppni 


TFXT 


1 characters 


i o n?Trn? 

lo V^XVC#J-/Z# 


TFXT 


1 characters 

^ VXAi*l t»V wWX w 


10 r^p'PFi^ 

xy \^i\JZUO 


TFYT 

X Xj*jc\. X 


1 characters 


ZU l^xxXlX-JH 


TFYT 


1 characters 


Z, 1 lN KJy^lSJLZiXJ A 


TFXT 

X Xw/\ X 


1 characters 


77 NOPRFD2 


TEXT 


1 characters 


J/N V^/v^lVXLJL/*? 


TFXT 

X lw/V X 


1 characters 


24 NOCRED4 


TEXT 


1 characters 


25 HLDEGALL 


TEXT 


2 characters 




TFXT 

X JU/V X 


2 characters 


27 LTVEPLAN 


TEXT 


1 characters 


28 LTVEPREF 


TEXT 


1 characters 


?Q CHOICE 


TEXT 


1 characters 


30 DISTHOME 


TEXT 


1 characters 


31 NUMAPPLY 


TEXT 


1 characters 


32 NUMACCPT 


TEXT 


1 characters 


33 ATDOl 

* 
* 


TEXT 


1 characters 


* 
* 

290 RESPRACE 


TEXT 


1 characters 



291 NORMSTAT TEXT 1 characters 
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ELECTRONIC SURVEY DATA ON GOPHER 

Cynthia B. Lucia 
Staff Assistant, Information Services 
SUNY Potsdam College 

A common chore of IR offices everywhere is filling out surveys 
such as College Board, Peterson's Guide, and Barron's. These surveys are 
usually completed, then forgotten until the next year. Yet how a college 
responds to these surveys could affect its image among high school 
students and counselors or even if the college gets lauded by Money 
Magazine or U.S. News and World Report. 

At Potsdam College we are using client-server "freeware" called 
Gopher to "publish" the survey responses to the wider college campus 
community and beyond. We include both the quantitative data and the 
narrative about aspects of the college that is often asked for in the 
surveys. 

By making survey response data accessible electronically to our 
campus community, it allows other offices to be more efficient in 
information-gathering, as well as providing consistent information. At 
present, four of our larger surveys (College Board, Peterson's, Barron's, 
College Counsel) have been loaded onto our Public Server which resides 
on the campus-wide network (LAN). From these surveys, we developed 
an electronic mini-fact book of statistics and narrative. The narrative is 
a composite of information taken from numerous College catalogs and 
brochures. 
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This mini fact book is available on Potsdam's Gopher and is now 
accessible to anyone who cares to look, anywhere in the world, with 
access to Gopher. It is located under Academics and Administration, 
Departments and Offices, Information Services. Within that file there 
are 21 separate files by subject, such as Admissions, Financial Aid, 
Undergraduate Majors, Athletics, Computer Services, Libraries, and so 
on. There is also the ability to "archive" the files from one year to the 
next, so that data could be compared for trend indicators. 

Our campus is multi-platform, and although most faculty, students 
and some of the administrative offices use MAC, our library, several 
academic departments, our business office, and human resources use 
DOS. Some areas use both. Our Public Server, accessible through the 
campus network, is available to MAC users only, and thus limits what 
DOS users can access. Gopher shows promise in providing a solution to 
our multi-platform compatibility problems. 

Formatting of material to be networked is another factor to be 
considered. Everything loaded on Gopher must be in text, thus tables, 
charts, and spreadsheets cannot be made available. The mini fact book 
is designed to be converted to text and is suitable for Gopher. However, 
our surveys, even using text with layout, do not convert to a readable 
format. Consequently, the mini fact book is on both Gopher and our 
local Public Server, and the surveys are on the local Public Server only. 

Knowing that survey responses might impact recruitment, public 
relations, or whether we get chosen by US News, how often do we get 
involved with or seek input from college relations offices, deans, vice 
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presidents, admissions officers, and career services. Our plan is to send 
email to these offices, maybe once a year or so, and ask them to check 
into our networked folder to review what we are saying about Potsdam 
College. 

Publishing this information on the Gopher server has three 
advantages: first, anyone who is writing a report, making a speech, or 
completing another survey has this wealth of information that they can 
pull down onto the wordprocessor on their desktop computer; second, 
the IR office can, with minor editing, complete next year's survey in far 
less time; and third, the quality of the narrative information is much 
enhanced due to the multiple perspectives contributing to the prose. 
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"How Insensitive Can You Get?" 
Further Analysis of the Best Colleges Rankings 



Michael McGuire 
Franklin & Marshall College 



Abstract 



■ERIC 



At the 1992 NEA1R conference, the author presented a detailed critique of the 
methodology that U.S. News employs in its annual Best Colleges rankings of 
colleges and universities, as well as the results of a survey on the weights used in 
those rankings. As a sequel to the 1992 discussion, the present session will feature 
the results of sensitivity analyses of the 1992 Best Colleges data for the National 
Liberal Arts category. Shifts in institutions' ranks were observed under three 
alternative weighting scenarios constructed from data obtained in a 1992 survey of 
college presidents, admissions directors, and academic deans. The implications of 
the Best Colleges model's instability for both the institutions affected by those shifts 
and the consumer audience w T ill be discussed. 



Introduction 

The annual rankings of colleges and universities in the Best Colleg es 
publication of U.S. News and World Report have prompted ample criticism from 
the higher education community. These criticisms cover a wide spectrum of issues, 
among which can be found the following: 

• There is no absolute "best" college — the fit between a given student's needs 
and an institution's strengths will determine which college is best for him/ 
her. 

• The measures used to define quality in the Best Colleges model tend to be 
weighted heavily toward wealth and prestige, deflating the rankings of less 
well-known and well-endowed institutions. 

• The model is even less valid for institutions with non-mainstream 
missions. 

• The measures used in Best Colleges are input- rather than process- or 
output-oriented; they are also reductionistic, blurring important nuances 
and ignoring the importance of an institutions gestalt. 

• The scales used in Best Colleges are crude, and the weights assigned to 
measures are arbitrary. 
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• Falsification of data is believed to be commonplace, further reducing the 
validity of the resulting rankings. 

• The U.S. News method places an uninvited and unwelcome data collection 
burden on participating institutions. U.S. News make significant profits 
from Best Colleges without compensating institutions for their assistance. 
Institutions that refuse to provide data for Best Colleges run the risk of 
being misrepresented, and potentially disadvantaged, in the rankings. 

A study conducted at Franklin & Marshall College in 1992 attempted to shed 
further light on the validity of the weights used in the Best Colleges model and to 
elicit suggestions for improving it. Results indicated a moderate level of 
disagreement between the U.S. News weights and those recommended by college 
presidents, admissions directors, and academic deans. Since the U.S. News weights 
change from year to year, and since even ideal weights (however they might be 
determined) would not compensate for the other shortcomings of the model and its 
underlying premises, this disagreement may or may not be meaningful. 

Of perhaps greater significance is the potential impact of different weights on 
an institution's ranking. Indeed, if different weights yield the same rankings, then 
the use of arbitrary ones might even be justifiable since they are by definition more 
efficient to formulate than empirical ones. If, on the other hand, different weights 
yield different rankings, the conscious use of arbitrary weights becomes 
problematical and may undermine the integrity of those rankings (other objections 
aside). The purpose of the present study was to seek an answer to this question by 
examining fluctuations in the Best Colleges rankings as a function of changes in the 
weighting scheme. 



Method 

Three alternative weighting schemes were applied to the 1992 Best Colleges 
data for National Liberal Arts colleges, supplied by U.S. News. The alternatives 
were derived from the aforementioned Franklin & Marshall study of weights 
recommended by college presidents, admissions directors, and academic deans. 
Weighting scheme 1 used the average recommended weights from the Franklin & 
Marshall study; the weights for schemes 2 and 3 were 0.5 standard deviations from 
those average weights, and counterbalanced so the total weights within a dimension 
totalled or approximated 100% (see Table 1). 

The overall score and rank for each of the 140 National Liberal Arts colleges 
were recomputed using the alternative weighting schemes. The new ranks and 
quartiles were then compared to those derived using the U.S. News weights to 
determine the impact of variations in those weights — the "bottom line" of Best 
Colleges, from an institutional perspective. The sensitivity of the U.S. News 
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Table 1 

Weights of Different Models 







Alt Model 1 


Alt Model 2 


Alt Model 3 






1992 US News 


(1992 study) 


(+/- .5 sd) 


(+/- .5 sd) 


sd 


Selectivity 












High school class rank 


5o% 


A 1 CI 

41% 


O A 07 




14/0 


Acceptance Rate 


20% 


15% 


10% 


20% 


11% 


Average SAT 


35% 


29% 


35% 


23% 


12% 


Yield 


10% 


16% 


21% 


10% 


11% 


Financial Resources 












Educational expend/ student 


OU 1 70 


used weignt 


used vv eigne 


usea vvcignc 




Other expend/student 


20% 


from US News 


from US News 


from US News 




Instructional Resources 












n-i 1 « * "11.* 

% Faculty full-time 


20% 


lo% 


20% 


11% 


Oct 

9% 


% Faculty with PhD 


30% 


28% 


23% 


32% 


9% 


Faculty - Student ratio 


20% 


39% 


44% 


34% 


10% 


Average faculty salaries 


30% 


18% 


13% 


22% 


9% 


Retention 












Alumni giving rates 


33% 


32% 


25% 


39% 


14% 


5-year Graduation Rates 


67% 


68% 


76% 


60% 


16% 


Overall 












Selectivity 


25% 


22% 


27% 


17% 


9% 


Instructional Resources 


20% 


26% 


22% 


30% 


8% 


Financial Resources 


15% 


21% 


17% 


25% 


8% 


Retention 


15% 


19% 


24% 


14% 


9% 


Reputation 


25% 


13% 


9% 


17% 


8% 
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rankings - as defined here, their susceptibility to significant fluctuation as a result of 
relatively minor and empirically derived deviations from the published weights — 
was thus measured. 



Results 

The results of this analysis are presented in Table 2. The sensitivity of the U.S. 
News rankings can be measured at a fine (i.e., shift in specific individual rank) or 
gross (i.e., shift in quartile, and thus Best Colleges cluster) level. A large percentage 
of colleges shifted one or more specific ranks in each of the 3 alternative models. 
Not surprisingly, those ranked originally in the 2nd and 3rd quartiles — on either 
side of the median — shifted more dramatically than those in the 4th and especially 
1st quartiles. Colleges shifted an average of 5 positions using empirically derived 
weights that were typically only slightly different from those employed by U.S. News 
(Table 1). Two institutions shifted upward (i.e., improved in rank) by 24 positions 
when alternative weights were used; for them, obviously, the U.S. News weights are 
not fortuitous compared to the tested options. On the other end of the continuum, 
two colleges shifted downward (i.e., declined in rank) by 22 positions when 
alternative weights were used. Those institutions seem clearly to be favored by the 
U.S. News weights. 

In terms of practical impact, only shifts in specific rank among the top 25 
institutions are significant, since those are the only specific ranks that U.S. News 
publishes in Best Colleges . There was considerable movement within that group of 
colleges under all 3 alternative weighting schemes, though 24 of the 25 would have 
remained on this prestigious list under all scenarios. An average of approximately 
22 colleges (16%) would have shifted into different quartiles and different sections of 
the published rankings, however, when the entire group of 140 institutions is 
considered. An alternative weighting scheme would have substantially benefited 
around 11 colleges, and disadvantaged 11 others, on the average. 



Discussion 

The present study attempted to determine the impact on institutional rankings 
of weighting schemes that were systematically different from the one used by U.S. 
News in Best Colle ges. The alternative schemes were derived from a study on 
optimal weights and were based on the input of dozens of top administrators at the 
liberal arts colleges in question. In the final analysis, the alternative weights had a 
clear and consistent impact on the resulting rankings, for both specific ranks and 
quartiles. It does in fact matter what weights are used, which makes the arbitrary 
nature of the weights in the Best Colleges model all the more disturbing. 

In many ways it is not surprising that the rankings are sensitive to fluctuations 
in weights; the latter are integral to the calculation of the overall score that 
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Model 1 

Quartile 

Top 25 

Rest of 1st 

Second 

Third 

Fourth 

Total 



Model 2 

Top 25 
Rest of 1st 
Second 
Third 
Fourth 

Total 



Model 3 

Top 25 
Rest of 1st 
Second 
Third 
Fourth 

Total 



Composite 

Top 25 
Rest of 1st 
Second 
Third 
Fourth 

Total 
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Table 2 
Shifts in Rank and Quartile 





Changed Rank 


Mfean shift 


Largest 


Largest 


Changed Quartile 


N 


# 


% 


in rank 


upshift 


downshift 


# 


Direction 


25 


20 


80% 


1.52 


5 


A 


1 


down 


If) 


9 


90% 


3.40 


8 


-10 


3 


1 up, 2 down 


35 


33 


94% 


6.20 


15 


-15 


6 


2 up, 4 down 


35 


30 


86% 


5.37 


24 


-16 


6 


4 up, 2 down 


35 


28 


80% 


3.83 


15 


-15 


2 


up 


140 


120 


86% 


4.36 


24 


-16 


18 





13% 



25 


18 


72% 


1.16 


10 


8 


80% 


3.00 


35 


32 


91% 


6.14 


35 


35 


100% 


6.66 


35 


30 


86% 


5.20 



140 123 88% 4.92 



4 


-3 


1 


down 


9 


-8 


3 


1 up, 2 down 


9 


-22 


7 


2 up, 5 down 


24 


-17 


8 


5 up, 3 down 


24 


-22 


3 


up 


24 


-22 


22 





16% 



25 


20 


80% 


1.76 


10 


9 


90% 


4.50 


35 


33 


94% 


7.17 


35 


32 


91% 


5.37 


35 


31 


89% 


3.97 



140 125 89% 4.76 



5 


-5 


1 


down 


8 


-10 




1 up, 2 down 


21 


-21 


8 


2 up, 6 down 


22 


-19 


9 


6 up, 3 down 


13 


-7 


3 


up 


22 


-21 


24 








17% 





25 


19 


77% 


1.48 


5 


-5 


1 


down 


10 


9 


87% 


3.63 


9 


-10 


3 


1 up, 2 down 


35 


33 


93% 


6.50 


21 


-22 


7 


2 up, 5 down 


35 


32 


92% 


5.80 


24 


-19 


8 


5 up, 3 down 


35 


30 


85% 


4.33 


24 


-22 


3 


up 


140 


123 


88% 


4.68 


24 


-22 


21 
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determines the rankings. What is surprising is that this issue has not been 
researched before, with empirically justifiable changes in the U.S. News weighting 
scheme a logical consequence. 

A related problem arises from the fact that U.S. News changes its weights 
periodically (though such changes are apparently not in direct response to research 
findings). Since changes in weights produce changes in ranks, all else bring equal, it 
is possible for a college to climb or slip in the Best Colleges rankings without any 
changes in institutional characteristics or the statistics that U.S. News uses to 
measure those characteristics. Institutions have an understandably haid time 
explaining such slippage to internal and external audiences when the institution's 
behavior, and its absolute and relative "quality" measures, remain unchanged. The 
proviso that consumers should not attempt year-to-year comparisons of rankings 
because of this instability in the model ignores the fact that many consumers will 
insist upon doing just that. 

As stated in the Introduction, this problem with weights is only one of many 
that plague the U.S. News methodology. While a non-arbitrary weighting scheme 
would almost certainly benefit that methodology and improve the credibility of Best 
Colleges among knowledgeable consumers and college administrators, it would not 
address the myriad other concerns that have been voiced repeatedly about that 
report. The statistical sensitivity of the Best Colleges rankings to relatively minor 
fluctuations in the study's weights has now been demonstrated; in spite of overtures 
to the contrary, the sensitivity of its authors to the many other objections of the 
higher education community has not. 
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ABSTRACT 

Three quasi-experimental studies are presented which lend support for the 
positive effects of developmental reading courses on reading comprehension levels of 
junior college students. Study 1 used a regression-discontinuity design to test the effects 
of the developmental reading course on overall grade point average. Standardized 
reading test scores were regressed against overall grade point average for students whose 
reading test scores either placed them in a developmental reading course or above the 
cutoff point requiring no placement. The regression findings for the classification 
variable showed a significant effect for group suggesting a positive direct effect of the 
developmental reading course on overall grade point average. Possible "mortality" bias 
was ruled out in Study 2 using a nonequivalent control group design. Study 3 used a 
single group pretest-posttest design to assess the effectiveness of the developmental 
reading course on improving reading comprehension skills. Significant pre- to posttest 
gains were found. These results form a critical multiplism indicating a positive overall 
effect for the developmental reading program. 
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A Critical Multiplisl Evaluation of Developmental Reading 
Instruction at Suffolk Community College 
To meet the needs of academically H under-prepared M entrants, colleges and 
universities throughout the country have implemented basic skills assessment and 
remediation programs. The goals of these programs are first to identify skills 
deficiencies and then, where indicated, to attempt to elevate reading, writing or 
mathematics skills proficiency to levels commensurate with the demands of college-level 
coursework through developmental or remedial courses. Currently, there is conflicting 
evidence on the effectiveness of these programs. 

In a review of the reading skills assessment programs at three community 
colleges, McElroy (1985) reported that students completing a remediation program had a 
higher graduation rate than those students who were either diagnosed as not needing 
remediation and those who did not complete remediation. However, McElroy (1985) 
also found some conflicting results which complicate the relationship between 
assessment, remedial programs, and student success. At one community college, 50 
percent of the students who did not complete the remedial reading course received a 
CPA of 2.0 or higher, while at another community college there were no differences in 
GPA or in credits earned to credits attempted between those completing and those not 
completing the remedial reading course. 

Similar conflicting results have been found by Hodges (1981) at another 
community college. Students placed in the remedial reading classes did not necessarily 
achieve higher grades than others. These remedial classes seemed to benefit the poorest 
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students the most (61 percent of those enrolled in the remedial classes received a 2.5 or 
higher GPA whereas only 40 percent of those students who did not enroll in the 
remedial course did as well). For the students who were below average, but not among 
the poorest readers, the reading course did not help in their chance of attaining a C 
average, but did help 10 percent of these students to achieve a B average. It was 
concluded that improved reading ability does not alone increase student success in other 
courses. 

The developmental education program at Suffolk Community College (SCC) 
focuses on remediating reading, writing, and math skills to college levels. As part of the 
outcomes assessment orientation and mission of the college, the developmental studies 
program is continually subjected to ongoing evaluation efforts to measure its 
effectiveness. This paper describes a recent series of studies which focus on the 
developmental reading program at SCC. 

The process of selecting courses at SCC follows the administration of a series of 
placement tests to assess basic skills levels. For the past five years the college has used 
the College Board's Computerized Placement Tests (CPTs; College Board, 1990) to 
identify students in need of reading remediation. Data on the SCC population reveal 
that the CPT Reading Comprehension Test (CPT-Read) has served as a reliable and 
valid assessment device for identifying students in need of some level of reading 
remediation. Specifically, the test developers report test-retest reliabilty to be equal to 
.90. for a sample of eighteen hundred examinees (College Board, 1990). Napoli. (1991) 
and Napoli and Coffey (1992) obtained validity coefficient ranging from .50 to .63 when 
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vailiditating the CPT-Read against other standardized measures of reading 
comprehension and performance in college level coursework 

The developmental reading program at SCC consists of a sequence of two single- 
semester reading courses. Ine first course (Introduction to College Reading, RE09) was 
designed for students with low-level reading abilities - that is, those with CPT-Read 
scores below 66, which has been equated to reading abilities below the eighth grade 
level (Napoli & Coffey, 1992). These students attend two semesters of developmental 
reading instruction. Students with assessed reading levels below college level but, above 
the eighth grade level (CPT-Read snores between 66 and 72), are only required to 
attend the second developmental course (Reading in the Content Areas, RE10). 

The RE09 course description appearing in the SCC catalog, states that the 
objectives and goals are "to provide individual and small-group instruction in basic 
reading and study skills in order to develop a higher level of competence so as to assure 
success in subject classes and allow entry into RE10." The course description for RE10 
includes the following statements: "designed for the student who needs to enhance basic 
reading skills necessary for successful completion of other content-area courses by 
developing students' ability to: read and study textbook materials effectively and 
discover main ideas in paragraphs; discover meaning through the use of absolute and 
conditional language; note details and make inferences; recognize structural devices in 
sentences and paragraphs; draw conclusions; outline and summarize; take notes from 
written and oral material; use proper form and style for research paper writing; develop 
vocabulary; prepare for and take exams; and develop study skills.' 1 
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Remediation is achieved by a combination of factors. Class size in developmental 
education courses is limited to eighteen students, thus providing individualized attention 
and follow-up. In addition, developmental education students spend at least one hour 
per course per week in the academic skills center, where they receive one-to-one 
instruction and work with self -paced books, tapes and computer software programs. 

The degree to which the developmental reading courses are achieving the desired 
level of remediation and fulfilling their objectives is the central focus of an ongoing 
evaluation. Since only quasi-experimental methods (Campbell & Stanley, 1966; Cook & 
Campbell, 1979) are appropriate to such an evaluation, a "critical multiplist" (Shaddish, 
Cook, and Houts, 1986) approach involving a combination of designs was employed to 
rule out rival hypotheses. This evaluation focuses on three recently concluded studies 
which were designed to assess the effectiveness of the developmental reading program. 
The first uses a "regression-discontinuity" design (Trochim, 1984) to examine the impact 
developmental reading courses have on a commonly accepted measure of college 
success, namely overall grade point average (GPA). The second study employs a 
nonequivalent control group design (Cook & Campbell, 1979), and compares 
developmental reading students to a sample of students with similar reading levels but 
who had not participated in the program. Study 3 employs a single group pretest- 
posttest design (Cook & Campbell, 1979) employing a standardized reading test. 

Study 1 

Sample 
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For the first study, two student cohort groups were identified from the college- 
wide master student data file. All students were selected from Fall 1988 through Fall 
1991 entrants. One group consisted of those students who scored below the "cutting 
point" (i.e., CPT of 80) and were enrolled in a developmental reading (Dev.Read) 
course (n = 6433) at any time during the semesters of Fall 1988 through Spring 1992. 
The second group consisted of students whose CPT-Read Score was above the cutting 
point and who "tested out" of the developmental reading program (n = 6109). We refer 
to these students as the non-developmental reading (Non-Dev. Read) group. 

Trochim (1984) provides an extensive review of how regression-discontinuity can 
serve as a design for program evaluations when randomized assignment is not feasible or 
possible, and where placement into a treatment occurs only for those subjects who fall 
on one side of a predetermined cutting point on a continuous interval scale. Seaver and 
Quarton (1976) employed the regression-discontinuity model to examine the effects of 
dean's list awards, a non-randomized assignment, on grade point averages (GPA) in 
subsequent terms for a sample of college students. These investigators regressed term 2 
GPA on term 1 GPA for dean's list and non-dean's students. The intercept at the dean's 
list cutting point was significantly higher than that of non-dean's list students who fell 
below the cutting point. This rise in the regression line at the dean's list cut point 
indicates the subsequent improvement or gain in academic performance associated with 
the award. 

Earlier work with the CPT-Read test has shown it to be reliably related to end- 
of-term course grades (Ward, Kline, & Flauger, 1986). More recently Napoli, (1991), 
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observed for a sample of 1,450 community college students significant correlations 
between the CPT-Read and Introductory Psychology final course grades (r = .52) and 
overall grade point averages (r = .41). If reading abilities could be significantly 
improved among program participants then their overall GPA would also be expected 
to shift or increase causing a regression-discontinuity between program participants and 
non-participants. Conversely, if the program has no effect on reading skills then the 
regression line for the program participants should not be displaced away from that of 
the non program students. 

Results for Study 1 

Statistical assessment of the CPT Read - GPA relationship, and an examination 
of potential regression-discontinuity between the student groups was tested employing 
the SAS general linear model procedure (SAS, 1988), In the first regression model GPA 
was simultaneously regressed on CPT-Read and the two-level class variable (Group) 
consisting of assignment to Dev. Read and Non-Dev. Read groups. Results for the 
analysis are presented in Table 1. As seen in the table, CPT-Read scores serve as 
significant predictors of overall GPA, F(l, 12539) = 502, p.< .0001). Following 
Trochim, an examination of nonlinear higher order CPT-Read effects failed to produce 
any meaningful increments in R 2 . The regression findings for the classification variable 
detected a significant effect for group, E( 1,12541) = 493, p. < .0001. An examination 
of the CPT-Read X Group effect failed to produce any meaningful improvements in R 2 . 
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The significant main effect for the grouping variable indicates the presence of a 
significant regression-discontinuity (Trochim, 1984). To determine the nature of the 
regression-discontinuity, separate regression equations were created for each. In both 
cases CPT-Read was observed to be significantly related to GPA (see Table 2). Further 
examination of the regression constants show that the Dev. Read group has an intercept 
which is indeed higher than the Non-Dev, Read group. This difference, which is 
evidenced most noticeable at the criterion cut-point (see Figure 1), represents the 
regression-discontinuity between the two groups. 

Within the regression discontinuity design Pedhazur (1982) points out that testing 
the difference between intercepts (regression discontinuity) is the same as testing the 
difference between adjusted means obtained in an ANCOVa, A statistical comparison 
of differences between adjusted means (intercepts) was conducted employing post hoc 
mean comparisons of CPT-adjusted GPA means (Pedhazur, 1982). Adjusted mean 
comparisons were assessed employing SAS generated Duncan t-tests. Results for the 
comparison (see Table 3) indicate the Dev.Read students achieved a mean adjusted 
GPA significantly above the Non-Dev. Read students. 



Summary of Study 1 

An examination of relationship between initial reading levels and subsequent 
GPA for a group of developmental reading students and a group of non-developmental 
students, employing a regression-discontinuity model, shows that the GPA of 
developmental students is significantly greater than what their pretest scores would 
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predict. This finding suggests that involvement in the developmental reading program 
may be directly related to subsequent academic success as evidenced by the GPA gain. 
Numerous alternative explanations or threats to the validity of this assumption can be 
made, however. Chief among them is "mortality" or attrition (Campbell & Stanley, 
1966). In this regard, it is possible that there is significant attrition among the remedial 
reading students, such that only the brightest enroll in or complete the program. If this 
were the case, then we would expect these students to be better than students with 
comparable reading levels, but who are not exposured to the developmental reading 
program. To examine this rival hypothesis study 2 was conducted. 



Study 2 

To rule out the "mortality" or attrition bias a third student group was identified 
from the college's master student data file. This group (n = 2210) consisted of those 
students whose CPT-Read score was below 80 but who were not placed into or enrolled 
in a developmental reading course. These students based on the CPT-Read score, 
tested into the developmental program but for various reasons were not placed into or 
enrolled in a reading course. We refer to this group as the Placed/Non-Attender group. 
The purpose for creating the Placed/Non-Attender group was to serve as a 
nonequivalent control group for the developmental reading group (Campbell & 
Stanley, 1966). These students, based on their CPT-Read assessed reading levels, are 
equivalent to the Dev.Read group, with one important exception, they had not been 
exposed to the reading program (the treatment). It is therefore expected that, if the 
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developmental program is unrelated to future academic success, then it would have 
higher CPT scores than the Placed/Non-Attender's group that could be primarily 
attributed to mortality bias. Conversely, if the program is achieving its goals the Dev. 
Read group should be similar at pretest to the Placed/Non-Attenders, but should 
achieve significantly higher GPAs than their matched counterparts at post-test. 

Results for Study 2 

To assess initial reading level comparability between the two groups, CPT-Read 
pretest means were compared. Results for the comparison indicates that the Dev.Read 
students (mean = 67.1) and Placed/Non-Attender students (mean = 66.9) had 
comparable (t(8,641) < 1.0, p. = .76) initial reading levels. This is a critical finding since 
it indicates that prior to exposure to the remedial course-work, program participants (i.e., 
Dev.Read) and qualified non-participants (i.e., the Placed/Non-Attenders) had nearly 
identical reading levels. If the Dev.Read students had shown an initially higher reading 
pretest mean, then any subsequent between group GPA differences might be best 
attributed to an initial advantage among program participants or a selection bias, rather 
than programmatic factors. With nearly identical performances on the pretest measure, 
however, it appears quite justified to deploy the Placed/Non-Attenders students group as 
matched control group. 

A comparison of the GPA means for the two groups indicates that the Dev.Read 
students earned a significantly higher (t(8,641) = 20.7, p.<.000l) overall GPA in 



comparison to the Placed/Non-Attenders. The mean GPA for the Dev.Read students 
equals 2.40, whereas the mean GPA for the Placed/Non-Attenders' equals 1.93. 



Summary of Study 2 

The focus of the analyses presented above was to test for "mortality" bias as a 
factor contributing to the post-program academic achievements of the Dev.Read 
students. After identifying a matched control group, and confirming the success of 
matching, a comparison of GPA means for the Dev.Read students and the control 
group (i.e., Placed/Non-Attenders) revealed that the program participants earn 
significantly higher GPAs than control students. Since program participants achieve 
higher subsequent GPA in comparison to initially similar controls, however, we can rule 
out mortality bias as a plausible rival explanation. 



Study 3 

Results from Studies 1 and 2 suggest that exposure to the developmental reading 
program produces significant gains in academic performance among students who would 
otherwise perform at lower achievement levels. These long-term performance gains may 
be attributed to programmatic factors which enhance reading comprehension levels, but 
this conclusion requires a more immediate assessment of the Developmental Reading 
Program to be substantiated. From an evaluation perspective, determining the degree to 
which a remedial reading program accomplishes its immediate objectives, improving 
comprehension skills, can be achieved by: 1) identifying relevant parameters which 
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would be sensitive to skills development; 2) selecting a sample of students to serve as 
reliable representatives of the developmental population, and 3) obtaining both pre-and 
post-instruction measurements on the relevant parameters. To this end, it is the practice 
of the college to conduct periodic posttesting, employing the CPT-Read on randomly 
selected samples of developmental students, to assess the degree of skills growth over 
the course of the term. 

The rationale for selecting the CPT test to serve as a relevant parameter is 
supported by two lines of reasoning. First, (as stated above) , the test has sufficiently 
established reliability and validity to be accepted as a true measure of the construct 
(reading) it was designed to measure. Secondly, utilizing the same assessment tool on 
subsequent occasions allows for a direct assessment of change since the repeated 
administrations are made with the same measuring device. 

Sample 

The sample consists of 555-RE09 students, and 910-RE10 students attending 
randomly targeted classes between the Fall 1990 through Spring 1992 semesters. For 
both groups the pretest to posttest interval was approximately 16 weeks. 

Results for Study 3 

Data aggregated from the pretest and end of the term posttest administrations of 
the CPT-Read test (i.e., pretest to posttest comparisons for the two developmental 
reading courses) were examined. The results of the statistical comparisons (T-tests for 
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correlated groups) between pretest and posttest CPT-Read means broken-down by 
developmental course (RE09 & RE10) appear in Table 4. These results indicate that 
significant pre-to post-performance gains were observed in CPT-assessed reading 
comprehension levels for students in both RE09 and RE10 classes. 



Discussion 

Employing a psychometrically reliable and valid reading comprehension test, 
significant reading skills performance enhancements were observed over the progression 
of the remedial reading courses. A statistically significant improvement from pre-to 
posttest, however, cannot be automatically interpreted as a meaningful or substantial 
improvement. It is, rather, a prerequisite for such conclusions. Only by an examination 
of the level and magnitude of gain following the detection of statistically significant 
movement in group means can such conclusions be drawn. 

In a series of studies which focused on the criterion-related validity of the CPT- 
Read test (Napoli, 1991; and Napoli & Coffey, 1992), three points on the CPT-Read 
score distribution were identified to serve as reliable markers to: 1) identify students 
with reading comprehension levels commensurate with the demands of college-level 
course-work (CPT reading comprehension test scores above 72); 2) identify students 
with moderate comprehension-level difficulties who would benefit from RElO-level 
remediation (CPT scores between 66 and 72); and 3) identify students with more 
pronounced comprehension difficulties i.e., below eighth-grade reading level) who would 
be best placed into the RE09 format (CPT scores of 65 and below). 
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An examination of the mean posttest values, appearing in Table 4 shows that on 
average the performance of the students in both groups was elevated to a proficiency 
level sufficient either to 1) move into the next level of developmental courses, as in the 
case of the RE09 students; or 2) "test out" of the developmental reading program, as the 
average RE10 posttest score indicates. 

The findings from the three studies provide compelling evidence that exposure to 
the reading program produces meaningful enhancement in reading comprehension levels. 
It is unlikely that the observed reading performance improvements can be attributed to 
"mortality" bias since the results of Study 2 failed to detect that phenomena within the 
developmental population. Together with evidence provided from Studies 1 and 2 there 
appears to be convergent empirical or "critical multiplist" support to conclude that the 
developmental reading courses are indeed achieving their stated goals. Individually, 
none of these studies provides convincing evidence to link the remedial intervention to 
subsequent academic outcomes- A cntical multiplist approach, however, provided 
sufficient convergent evidence to assess the programs impact. 

As a final consideration, future evaluation efforts must continue to monitor the 
success of the developmental reading program in similar replication studies employing 
long-term outcomes. Ultimately, only through comprehensive longitudinal investigations 
can the full impact of the program be assessed. 
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Table 1 

Analysis of Variance for the Regression of G PA on CPT Read and Group, 



Source of Variance 


df 


MS 


F 


CPT-Read 


1 


390.2 


502.6* 


Group 


1 


383 


493* 


Error 


14,748 


.86 





* p.< .0001 



Table 2 

Regression of GPA on CPT-Read. for each GrQUft 



Group 


df 


F 


A - Intercept 


b.Re%rc5$ion Coefficient 


Non-Dev. Read 


1, 6107 


216.9* 


.614 


.021 ! 


Dev Read. 


1, 6431 


3653* 


1396 


.012 



* p. < # .0001 



Table 3 

Comparison o i CPT Adjusted GPA.Means for Developmental Reading^iudgnis_and 
Non Devel o pmental Students . 



Group (N w = 12542) 


Adjusted 1 Mean GPA 2 (SEM) 


Non-Dev. Read (n = 6109) 


239 (.016) 


Dev. Read (n = 6433) 


236 (.013) 



1 Means which are statistically adjusted for initial CPT-Read Levels. 

2 Ail adjusted means in column significantly differ at the p. < .0001 level. 
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Com parison of CPT-Read qml Nfe*™ ffir BJEQS and RElO College-Wid g 



Developmental 


Pretest 


Posttest 




Class/N 


Mean (SD) 


Mean (SD) 


Mean (Std Err) 








T Prob. < 


RE09 


50.5 (9.4) 


64 (14.8) 


13.5 (.63) 


N = 555 






21 ,0001 


RElO 


71.7 (8.9) 


75.5 (15.4) 


3.8 (.53) 


N = 910 






7,24 ,0001 



Figure 1. Regression of GPA on CPT-Read. for Dev. Read and Non-Dev. Groups. 
3.5 -j : 7 1 
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Introduction 

The challenges faced by academic administrations are varied and range from 
student academic experience to financial and budgetary concerns. Most of the issues 
faced by adminstators are not easily resolved due to their multi-faceted nature and to the 
complexity of academic environments. One approach to resolving administrative concerns 
and determining policy might be to rely on theoretical frameworks and research literature 
which exists in the field of higher education, and then to look to internal data, such as 
survey responses, retention patterns, etc. to establish more specific plans based on 
information unique to the institution. 

Institutional research offices often have access to institutional data which can be 
used to provide information for adminstrative questions. Because there are so many 
factors/variables which may potentially have influence in any situation, it is important that 
specfic effects can be isolated and measured. This paper will described how inferential 
statistical methodologies may be applied to existing data to address administrative issues. 
The benefits of such analyses in terms of information gained will be described, and the 
limitations in terms of statistical and practical value will be discussed. A specific example 
of how these methods were applied to an administrative project, and how results were 
interpreted, is given. 
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Background of Project 
A college committee was formed to address issues relating to the freshman 
academic experience at Ithaca College. One of these topics focused on the possibility of 
developing a program or campus center with the purpose of assisting students with 
academic skills, i.e. study habits, test taking strategy, time management, etc. Several 
questions could immediately be raised: Should students be required to participate? If not, 
toward which students should the program be targeted? How might one predict which 
students are likely to be at risk with no prior knowledge of the students college academic 
performance? Is needing help with academic skills independent of indicators of general 
academic ability? 

Use of Inferential Statistical Methods: 
Benefits and Limitations 
One way to provide information for questions such as these is to turn to statistical 
analysis of instituitional data. Such analyses can provide benefits in the information they 
provide, and yet may also be misleading if not interpreted appropriately. Additionally, the 
results of such analyses may be difficult to apply in practice. The following analyses will 
be used as an example in the discussion of the possible benefits of statistical analysis in 
providing useful information relevant to planning interventions, and as an example in 
discussing the limitations of such analyses in both statistical and decision-making 
frameworks. 

Several sources of existing institutional data were used to address the questions 
described above. A pre-semester questionnaire asked students to indicate if they would 
like to receive help with academic skills. Admissions data, including SAT scores and high 
school class standing were also used. Finally, the QPA (quality point average) obtained by 
students at the end of their first semester was used in the analyses. 
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Benefits: Information gain from analyses. 

First a series of t-tests were performed comparing the first semester QPA's earned 
by those students who indicated that they would like to receive help with academic skills, 
and the QPA's earned by those who did not request help. Table 1 summarizes the results 
of these analyses. The results indicate that those students who requested help with 
academic skills earned a lower QPA (M = 2.779) than those students who did not request 
help (M = 2.929; t = 4.07, p < .01). This would indicate that students who requested help 
have accurately diagnosed their potential academic weaknesses; that is, they believed that 
they would need help, and the first semester QPAs indicated that they, indeed, may have 
benefitted from academic skill help. 

Further information is obtained when t-tests are performed on student 
subpopulations. When students are classified according to the five schools within the 
college (Table 1), the tendency for siudents who requested help to obtain lower first 
semester QPAs is present in every group with the exception of one (School B), and this 
one is not significant. The effect is significantly pronounced among students in School A 
(2.683 vs 2.830), School C (3.054 vs. 3.205), and School E (2.966 vs. 3.219). Excluding 
the possibility that these differences are not a function of these school categories, it may 
be useful to target these populations for any planned intervention, particularly if resources 
allow only limited intervention programming. 

Second, analyses were performed in order to identify other variables which may be 
causing these observed differences in QPA, and to assess how these other variables 
might interact with "Request Help" variable in predicting first semester QPA. For example, 
there is sometimes a relationship between SAT scores and college QPA (Pascarella & 
Terenzini, 1991), so one might expect that SAT scores may be exerting some effect on the 
QPA results obtained here. In order to assess the separate effects of SAT performance 
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and requesting help on QPA, students were categorized on the basis of their SAT scores 
and whether or not they requested help with academic skills. A series of ANOVA (Analysis 
of Variance) were performed, and the results are summarized in Table 2. 

Consistent with the results of previous analyses, a significant main effect for 
"Requested Help" versus "Did Not Request Help" was identified, with those requesting 
help earning a lower QPA than those who did not request help (2.929 and 2.779 
respectively; F = 5.55, p < .05). As expected, there was also a main effect for SAT score 
category, such that those students with higher SAT scores tended to earn higher QPAs 
than those students with lower SAT scores (F = 34.87, p < .05). One might hypothesize 
that SAT scores could be responsible for the differences in QPA obtained across help 
request categories; however, an ANOVA performed on QPA's and controlling for SAT 
scores, again indicated that students who requested help with academic skills earned 
lower first semester QPA than those who did not request help (F = 5.09, p < .05). 

One might also expect that a students class standing upon graduation from high 
school (rank in high school class) would also show a relationship with first semester QPA 
in college. In order to determine if this was the case, and to determine if this relationship 
was responsible for the differences in QPA across help request category, several analyses 
were performed. First, students were categorized on the basis of their high school class 
rank percentile and whether or not they requested help with academic study skills. Next, a 
series of ANOVA were performed; the results of these analyses are summarized in Table 
3. 

As expected, there was a significant main effect for high school rank such that those 
students finishing higher in their class tended to earn higher first semester QPAs (F = 
78.31 , p < .05). Consistent with previous analyses, there was also a main effect for help 
request category, such that students who requested help tended to earn lower QPAs 
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(2.801 ) than those who did not request help (2.997; F = 6.48, p < .05), and this tendency 
was still present when controlling for high school rank (F = 5.19, p < .05). Most interesting, 
however, is the significant interaction between heip request category and high school rank 
(F = 2.93, p < .05). In the two lowest categories of high school rank, those who requested 
help earned higher QPAs, while in the upper three categories those who requested help 
earned lower QPAs. This may indicate that those in lower percentile categories are 
underestimating their academic skills preparation. With respect to intervention strategies, 
it may suggest that students within the lower two quintiles who don't request help are more 
in need of assistance than those who do request help, although small within cell samples 
may limit interpretability in this analysis. 

So, the application of inferential statistical techniques has provided useful 
information which may be used in the planning and development of student programs. 
First, we have been able to target students who may be in need of academic skills 
assistance; that is, it has been determined that students can accurately diagnose their 
need for help, and that this tendency is more pronounced for specific college 
subpopulations. Second, we have identified other indicators of academic ability which 
have some effect on first semester QPA; and, students are still able to determine their own 
need for academic help when controlling for the effects of these variables. 

There are, however, numerous limitations in the application of such techniques. 
There are limitations in the information obtained from statistical analyses themselves, and 
there are also limitations in the application of results to administrative decision making and 
policy. 

Limitations: Statistical and Practical 

With respect to limitations in statistical inference, several concepts should be kept in 
mind. First, inferential methods are probability based and based on specific assumptions 
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regarding the distribution of events; they do not provide certain or absolute answers. A 
significant result with a p value of less than .05 merely indicates that a difference of the 
indicated magnitude would occur by chance in fewer than five of one hundred similar 
samples, assuming that the difference in true population means is zero (Mohr, 1990). 
Second, one's chances of obtaining statistical significance increase as sample sizes 
increase (Winkler & Hays, 1975). It is true that a larger sample will produce a distribution 
more similar to the population distribution; however, a very large sample size may produce 
statistical significance when practical differences in group means are negligible. 

There is also the issue of causality in the interpretation of results. In research of this 
type, there is little control over the variables of interest. As with archival or field research, 
one is often merely uncovering relationships among variables, rather than gaining the 
insight into causality of relationships which can be helpful in structuring interventions. The 
demonstration of relationships among variables does little to explain causal mechanisms 
among those variables. Finally, there are issues of generalizability. A sample should be 
randomly drawn from the population of interest, although in practice, this is often not the 
case. The extent to which this is not true lessens the generalizability of results obtained. It 
is important to make a determination regarding how representative the analysis sample is 
of the popoulation of interest. 

Second, there are also limitations in the extent to which information obtained from 
such analyses can be easily applied in the context of decision making and policy. For 
example, in the allocation of finanical and human resources toward developing an 
academic skills center for freshmen, an administrator may be interested in specific 
indicators of payoffs and losses associated with actions. The probabalistic nature of 
statistical indicators, as well as possible problems with small or unrepresentative samples 
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which may characterize research such as this, make a simple application of statistical 
results to resource allocation difficult. 

Ultimately, too, individual and institutionalized values and goals will play a 
substantial role in decision making and policy. While analyses such as these may assist in 
targeting students and indentifying variables, they provide no simple rule for application. 
For example, one might conclude from the analyses described above that students who 
request help, and are in the School A, School C, or School E are best able to identify their 
academic weaknesses, and are the most likely targets for intensive assistance in 
academic skills. Given this scenario, other students who may benefit from such assistance 
are automatically excluded. While the approach is consistent with one system of values, 
excluding some students from a potentially enriching and beneficial program may be 
inconsistent with another set of values. So, while statistical analyses provide a means to 
describe "what is", the application of such information will be based on "what ought to be" 
(See Kendler, 1 993 for a fuller discussion of this distinction). 

Conclusions 

In conclusion, inferential methodologies can provide useful information to 
administrators, and help to isolate pertinent variables and to measure constructs in 
concrete dimensions. Such information can lend itself to increased objectivity in decision 
making. On the other hand, care must be used in interpreting and applying the results of 
such analyses. Ideally, research results are interpreted in the larger context of an existing 
theoretical framework and previous research results; in practice, no such context may 
exist. In this vein, consideration of the limitations of statistical inference, and the values 
and goals of specfic organizations become increasingly important. 
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Table 1 . Request for Help with Academic Skills: Means, cell counts, and t-values for all 
students and within School. 



Population 


Help Not 


Help 


t-value 




Requested 


Requested 




All students 


2.929 


O 770 

2. / /9 


a 07** 




(593) 


(916) 




School A 


2.830 


2.683 


2.72** 




(264) 


(449) 




School B 


2.535 


2.548 


-.09 




(54) 


(102) 




School C 


3.205 


3.054 


2.19** 




(122) 


(127) 




School D 


2.906 


2.887 


.22 




(98) 


(163) 




School E 


3.219 


2.966 


2.37** 




(55) 


(75) 





**p < .05. 
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Table 2. Request for Help with Academic Skills: Means, cell counts, and F-values for 
SAT groups. 



SAT Help Not Help Total 

score Requested Requested 



<850 



850-949 



950-1049 



1050-1149 



1150+ 



2.571 
(17) 

2.654 
(80) 

2.801 
(189) 

2.977 
(164) 

3.248 
(139) 



2.614 
(63) 

2.567 
(220) 

2.705 
(273) 

2.881 
(217) 

3.173 
(141) 



2.605 
(80) 

2.590 
(300) 

2.744 
(462) 

2.922 
(381) 

3.210 
(280) 



Total 



2.929 
(589) 



2.779 
(914) 



2.838 
(1503) 



SAT main effect F=34.87, p < .05 

Help request main effect F=5.55, p < .05 

Help request controlling for F=5.09, p < .05 
SAT score 
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Table 3. Request for Help with Academic Skills: Means, cell counts, F-values for High 
School Rank percentile group. 



%ile Help Not Help Total 

Rank Requested Requested 



<20th 


1.184 


2.407 


2.184 




(2) 


(9) 


(11) 


21 -40th 


2.235 


2.356 


2.324 




(24) 


(71) 


(96) 


41 -60th 


2.582 


2.476 


2.510 




(70) 


(148) 


(218) 


61 -80th 


2.932 


2.759 


2.827 




(135) 


(207) 


(342) 


81st + 


3.259 


3.174 


3.214 




(230) 


(260) 


(490) 


Total 


2.997 


2.801 


2.879 




(462) 


(695) 


(1157) 



H.S. Rank main effect F=78.31, p < .05. 

Help Request main effect F=6.48, p<.05. 

Help Request controlling for F=5.19. p<.05. 
H.S. rank 

Help Request/H.S. Rank F=2.93, p < .05. 
interaction 
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Introduction 

The persistence of salary differences between men and women has been well documented 
over the last twenty years, and the focus of much of this research has been upon male and female 
faculty in academe. Early research in this area assumed a human capital perspective and examined 
differences in individuals' skills, experience and education (Johnson and Stafford 1974; Hoffman 
1976). More recent research has focused on structural sources of earnings disparities, particularly 
the segregation of female workers in certain occupations and industrial sectors (Baron and Bielby 
1984: Hodson 1984; Tolbert 1986). There has been a relative neglect in this literature of 
organizational-level analysis, even though most theorists agree that organizational structures have 
important influences on salaries earned (Stolzenberg 1978; Tolbert 1986). While acknowledging 
other determinants of salary policies and practices, the major focus of this research is on the 
contribution of organizational structure to gender-based salary differences in academic institutions. 

Conceptual Framework 

As Smart recently noted in his research (1991), the study of faculty compensation and gender 
equity has been guided by two theoretical perspectives: human capital theories and 
structural/functional approaches. Human capital theory encourages researchers to attend to those 
variables that reflect a person's utilitarian investment in educational credentials (such as doaoral 
degrees) and work experiences (such as research and administration) that yield a greater return or 
higher financial compensation (Johnson and Stafford 1974: Freeman 1976: Betz and Fitzgerald 
1987). Structural/functional approaches, on the other hand, focus upon occupational segregation of 
women into academic disciplines and institutions that have lower prestige and compensation levels 
(Finkelstein 1984; Tolbert 1986: Youn & Zeltennan 1988 ). 

Earnings differences between males and females result from complex processes operating 
through individuals, institutions and their external environments (Bridges and Berk 1974, 23 1). From 
this perspective, academic salaries are determined by the interaction between individual faculty 
characteristics and credentials and the organizational context in which salary decisions are made 
(Stolzenberg 1978; Tolbert 1986). Faculty with comparable credentials and experience are expected 
to earn equivalent salaries if employed by the same institution in the same discipline. In other words, 
the benefits associated with academic employment are equitable if distributed equally to comparable 
faculty. 
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Organizational structures are influential in determining salaries because structures provide the 
arenas for organizational decision-making and operate as independent influences on salary decisions 
(Ranson, Hinings and Greenwood 1980; Hall 1991). The structures of an organization are the more or 
less permanent arrangement of organizational members in positions that influence the role relations 
among them (Blau 1974; Khandvvalla 1978). This definition implies a division of labor and a 
hierarchy of authority that specifies, in varying degrees, the rules and regulations of how incumbents 
are to behave in work positions. Differences in structural characteristics become important contextual 
influences on organizational decision-making. As Hall (1991) observed, once the decision to expand 
an organization is made, the increased size influences subsequent decision-making and perceptions of 
alternatives available for responding to internal and external conditions and constraints. Thus, some 
variation in salary decisions is expected from differences in organizational structural characteristics, 
and organizations with different structural arrangements are expected to have variations in their 
internal salary allocation procedures. The wage attainment research provides theoretical and 
empirical support for the organizational determinants of salary differences 

Organizational Structural Characteristics and Salary Allocation Decisions 

Wage attainment theory and research suggests that organizational structural characteristics 
have important influences on salaries. Talbert and Bose (1977) applied Weber's bureaucratic 
characteristics and open systems theory to develop the notion that organizational structure influences 
salary variation. They reason that the higher the degree of environmental uncertainty involved in a 
job and the greater the degree of employee discretion, then the higher the employee's salary. Thus, to 
the extent that employing organizations differ in r , degree of work routinization, 
interorganizational salary variations are expected. The same reasoning is applied to organizational 
subunits, units that manage environmental segments representing high organizational dependence, 
such as finances or personnel, are likely to receive a larger share of organizational resources, 
including salary (Talbert and Bose 1977). The more subunits managing the environment, the largei 
the number of personnel needed for coordination and the greater the extent of horizontal 
differentiation, one form of organizational complexity. The greater the degree of complexity 
exhibited by an organization, the higher the expected organizational salary level. 

Stolzenberg (1978) relied on the organizational research linking size, work standardization, 
and structure. The greater an organization's size, the greater the work standardization, and the 
greater the effects of employee's schooling on their wages and achievement. The processes of wage 
attainment may be very different in large organizations from those in small or medium-sized 
organizations. 

Baron and Bielby (1984) argued that work arrangements and labor market structures interact 
to influence environmental dominance and organizational complexity. Organizations confronting 
complex environments are favored when they adapt through internal complexity (size, structure, 
technology). The more complex the internal structure, the more successfully an organization has 
dealt with it's environment. Thus, larger and more structured organizations are better able to 
dominate their resource environments and to populate central segments of the economy while smaller, 
less complex and more vulnerable organizations occupy its periphery. The reward structures of less 
insulated, less complex organizations may be governed more by labor market forces and concerns for 
efficiency. The most influential characteristic in shaping work arrangements is organizational size . 

Tolbert (1986) extends the environmental dominance and structural complexity perspectives 
by combining them with Becker's (1957) notion of discrimination as "preferences" (p. 227). Tolbert 
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reasons that more dominant organizations are better able to attract and compensate members of a 
preferred group because these organizations are less driven by competitive pressures and have greater 
slack resources. Tolbert's research indicates that academic discipline and organizational size and 
wealth are good predictors of salary differences between male and female faculty. Her research 
suggests that salary allocation decisions may be different for males a< d females in larger, more 
affluent institutions, supporting the theory that preferences may be easier to sustain in institutions 
considered dominant in relation to other organizations in their environment. 

Tolbert examined differences in average salaries for male and female faculty to reach these 
conclusions, but her study did not include a variety of relevant individual characteristics such as 
experience and highest degree earned. Thus, Tolbert was unable to documeni a clear link between 
organizational influences and salary disparities. The present investigation overcomes this limitation 
by including a wide range of individual human capital influences on salaries, in addition to various 
structural variables. Such research is needed if the above theories are to find support in academic 
organizations. 

Organizational Influences from Higher Education Salary Equity Studies 

In addition to the human capital and wage attainment research, a few studies of salary equity 
in higher education have collected data on structural and organizational level variables (Cohn 1974; 
Cox and Astin 1977), while others tave included organizational influences along with individual 
determinants in their salary research (Astin and Bayer 1972; Darland et al. 1973; Bayer and Astin 
1975; Barbezat 1987; Smart 1991). 

One of the first attempts at measuring gender salary- inequities, considering both individual 
differences and differences in work settings, was the Astin and Bayer study (1972). Their sample of 
almost 6,000 faculty was randomly drawn from a national sample of 60,000 faculty representing over 
300 institutions. The 33 salary predictor variables represented demographic characteristics, 
educational background, professional work activities, including productivity and characteristics of the 
employing institution. They obtained an R-square of .64 and found that the three most influential 
variables in determining academic salaries were rank, productivity, and type of employing 
organization. Women employed by large institutions made less money than comparable men at the 
same institutions, and women were more often employed at smaller institutions, thus accentuating the 
overall difference in salaries between males and females. When Astin and Bayer controlled for the 
individual differences in faculty credentials, they found a salary disparity of $1,040 (in 1968-69 
dollars) between males and females, 

Cohn (1974) argued that the observable differences in salaries by institutional type may be 
misleading since salary differences may result frjm some underlying organizational factors rather 
than from institutional type. He investigated 13 organizational variables, representing a larger class 
of factors affecting faculty' salaries in a national sample of 204 institutions. The significant influences 
on salaries included organizational size, per capita income of the state where the institution was 
located, type of control, and student/faculty ratios. A surprising result was that the coefficient for 
university versus other types of schools was not significant, implying that once the other factors were 
taken into account, it did not matter what an institution called itself as far as salaries were concerned. 

Similarly, in Cox and Astin's 1977 study of 1,300 organizations, the effects of being 
employed at a university were eliminated when organizational size was controlled. Their findings 
suggested that university faculty were paid more because they worked in large rather than small 
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institutions. In addition to organizational size, organizational wealth, selectivity and percentage of 
graduate students positively influenced levels of faculty compensation. Results indicated that the 
percentage of women on a faculty negatively influenced faculty salaries, probably mediated, in part, 
by the concentration of women in smaller and less selective institutions (Cox and Astin 1977). 

Darland et al. (1973) examined faculty salary differences for all faculty and separately for 
men and women by type of institution and by field using more than 25 demographic, education, 
experience, and productivity variables. Salary returns from experience were twice as great for men 
as for women. Men also gained twice as much from administrative activity than women (Darland et 
al. 1973). Controlling for qualifications and productivity, women tended to be underpaid on average 
by roughly Si 500 annually (in 1969 dollars). The amount of underpayment was more pronounced in 
research universities and in the biological and physical sciences where salaries tended to be highest 
and the number of women were few. 

Barbezat (1987) estimated the extent of salary discrimination at two points in time using 
national data from the Carnegie Foundation for 1968 and from the Roper Center for 1977. She found 
evidence that the magnitude of gender-based salary differentials may be declining. The regression 
results demonstrated a proportional salary advantage ranging from seven percent to thirteen percent 
for faculty employed by research and doctoral institutions over faculty members in the liberal arts 
colleges. 

Tolbert (1986) examined several organizational influences on salary in a national sample of 
309 public and private colleges and universities. She found evidence of occupational segregation of 
women into lower paying fields and institutions. Organizational size was the strongest predictor of 
salary levels for hoth males and females, but males benefited considerably more from working in 
larger institutions than did females. Similar effects were found for organizational wealth and 
selectivity. Tolbert's results indicate greater average salary differences between male and female 
faculty at larger, wealthier, and more selective institutions. 

Smart (1991) applied a causal model to investigate gender equity in academic rank and salary 
using 1984 data collected by the Carnegie Foundation. Employing several variables reflecting both 
the human capital perspective and the occupational segregation perspective, he examined the direct 
and indirect effects of gender on academic rank and salary. Human capital variables (such as career 
age and highest degree) and structural variables (such as male dominated discipline) both proved 
significant. Gender had significant direct and indirect effects on rank and salary, but the indirect 
effects were much greater. "These findings illustrate the central importance of career age, academic 
rank, and the degree of male domination of academic disciplines as the primary variables through 
which gender indirectly influences salary attainment."(p- 520) 

In a study of salaries at NASULGC institutions, Scott and Bereman (1992) documented the 
enormous influence of academic field on faculty' salaries. In 1990. the lowest disciplines (such as 
education, letters, foreign languages and fine arts) had average faculty salaries that were 20% to 30% 
lower than the highest paid [male dominated] disciplines (such as engineering, business/management 
and the sciences). 

In sum, the organizational, wage attainment, and higher education literature all lend 
theoretical and empirical support to the influence of organizational and structural variables on salaries 
earned. Yet few recent studies have attempted to measure the size of the gender disparity attributable 
to organizational variables, while controlling for relevant individual measures. 
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METHODOLOGY 



The present investigation of organizational influences on salary disparities proceeds in two 
phases, the first de elops the individual and organizational predictors of academic salaries and the 
second phase examines their roles in predicting salary differences using single equation regressions. 
This study draws upon a 1987 database of 6,536 full-time faculty, 5,106 men and 1,430 women, in 19 
institutions within a single system of public higher education: two universities without medical 
schools, two universities with medical schools, seven comprehensive colleges, two baccalaureate 
institutions, two medical schools, and four multiprogram two-year institutions. 



There are both advantages and limitations in having sample organizations from the same state 
university system. While this limits the organizational and geographical diversity of the institutions, 
it also reduces unwanted variability in faculty compensation, such as fringe benefit packages and cost 
of living salary increases. Thus, any salary differences in this study are more likely to reflect true 
differences in real compensation than is usually the case in such studies. However, the effects of 
organizational isomorphism are expected to produce structural similarities among the sample 
institutions, thus reducing organization-based male/female salary differences. 

Another limitation of the study is the lack of productivity measures for either teaching or 
research. To the extent that men and women collectively differ in their academic productivity or any 
other salary determinant omitted from the analysis, then some of the variance attributed to the 
independent variables, including sex, may be a function of these factors. However, it seems 
reasonable to assume that, in the aggregate, research and teaching productivity are distributed equally 
among comparable academic males and females. 

If rank or any of the individual variables used to predict salary are biased, then the estimates 
of salary inequities in the present analysis are conservative, other things being equal. This condition 
appears likely since prior research indicates that academic women are promoted less frequently, 
offered less initial rank and less initial salary than men of similar credentials (Astin and Bayer 1972; 
Bayer and Astin 1975; Johnson and Stafford 1977: Ferber and Green 1982: and Smart 1991). 

Phase One: Variable Development 

Individual Determinant:, 

We examined data collected by the state university system and identified an array of 
demographic variables, educational credentials, and work experience measures that were employed in 
the studies cited above. These include each faculty member's gender, age, race, highest degree 
earned and years since highest degree, length of annual contract, initial rank, current rank and years in 
current rank, past and current administrative experience including department chair status, total years 
of full-time and part-time service at the current institution, and total years of professionally relevant 
experience outside the current institution. The work experience variables, in particular, are rather 
comprehensive when compared to wage attainment literature and much of the salary equity research. 



Limitations 
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We classified each academic field by HEGIS discipline, rather than using the Biglan 
classification (1 973) or some other framework. We chose to examine the effects of HEGIS discipline 
on salaries because of its relevance to the public university system under study. (See Appendix A for 
the HEGIS Discipline groupings and note that Psychology is included with the social sciences.) 

Organizational Size and Wealth and Complexity 

Since wage attainment research found strong effects from both organizational size and wealth 
associated with salary differences (Stolzenberg 1978; Hodson 1984: Tolbert 1986) and suggested the 
importance of organizational complexity (Talbert and Bose 1977; Baron and Bielby 1984), this study 
of gender-based salary differences utilizes organizational size, affluence and complexity as measures 
of an organization's dominant position and predictors of male/female salary differences. The 
organizational literature refers to size almost exclusively as the number of organizational employees. 
In a critique of the organizational size and structure literature, Kimberly (1976) finds that more than 
80 percent of the studies reviewed used the number of personnel as the only indicator of 
organizational size. However, the higher education literature frequently uses student enrollment as 
the preferred indicator of organizational size. The present study uses faculty FTE and student FTE as 
measures of organizational size, rather than relying solely on one or the other. 

Organizational wealth appears conceptually distinct from student enrollment or faculty size, 
since it refers to the discretionary resources rather than the organizational members available to the 
institution (HalL 1991). Blau (1973) conceptualized size and wealth as distinct in his analysis of 
faculty salaries in a national sample of academic departments. Volkwein (1986) found that student 
enrollment and faculty size occupied a different factor than various financial and wealth measures in 
his study of public universities. Moreover, both Hodson (1984) and Tolbert (1986) found that 
organizational wealth influenced the internal salary allocation processes differently for males and 
females. The present study estimates the differences in organizational wealth by three 1987 
variables: total educational and general expenditures and transfers per FTE enrollment, total 
sponsored funds research expenditures per FTE enrollment, and total library expenditures per FTE 
enrollment. 

There is some controversy in the literature regarding the relationship between size and 
complexity. (Blau and Schoenherr 1971 : Blau 1973; Hall 1991; Hall, Haas, and Johnson 1967: 
Argyris 1972) In his study of public research universities. Volkwein (1986) found that measures of 
size and complexity occupied the same factor. The structural concepts of size and complexity are 
often related, but since organizational complexity is reflected by the division of labor, job titles, 
multiple divisions and hierarchical levels, complexity appears conceptually distinct enough to warrant 
separate analysis (Hall, 1991). 

The greater the number of occupations and the longer the period of training required, the 
more complex the organization (Hage 1965). Academic institutions vary in the number of 
specializations offered and the length of time required for faculty training in these specialized areas. 
Consequently, these institutions differ in complexity as measured by the number of degree programs 
and the highest degree offered. Another dimension of complexity along which academic 
organizations can vary includes horizontal differentiation, the number of different subunits and 
divisions (Blau and Schoenherr, 1971: Hall, 1991). The larger the number of academic colleges, 
schools and academic departments on campus, the greater the horizontal differentiation, increasing 
the need for vice-presidents and deans and other coordinating personnel. Structural complexity in the 
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present study is estimated by three variables: highest degree offered at an institution, the number of 
department chairpersons and the number of vice-presidents and deans. 

An examination of the intercorrelations among the organizational variables revealed that 
several organizational variables are highly related, so a principal components factor analysis was used 
to reduce variable redundancy. Table 1 shows the results of the factor analysis which condenses the 
data from the original eight variables to three organizational factors: wealth, size and complexity. 
These factors retain 96.5 percent of the original variability and are superior for subsequent analysis 
due to their reduced intercorrelations. Two of the original three complexity measures, highest degree 
offered and number of department chairs, loaded on the size and wealth factors, indicating a closer 
degree of association with organizational size and wealth than with complexity. 

TABLE 1: Factor Analysis 



Organizational Variables 


Factor 1 
Wealth 


Factor 2 
Size 


Factor 3 
Complexity 


Research Expend./FTE Student 
Ed. Expend./FTE Student 
Library Expend./FTE Student 
Highest Degree Offered 


.9968 
.9899 
.9552 
.6627 


.5685 




No. of Department Chairs 
FTE Enrollment 
FTE Faculty 




.9699 
.9640 
.9609 




No. of Vice-Presidents + Deans 






.9608 


Percent of Unique Variance 
Joint Variance 


48.8% 
48.8% 


38.0% 
86.8% 


9.7% 
96.5% 



Note: Factor loadings are rotated (varimax) and loadings below .30 not shown. 



The organizational measures were averaged and summarized by NCES institution type 
revealing that the two medical schools exhibit the highest organizational wealth, the doctoral 
universities with medical schools are the largest in size, and the doctoral universities without medical 
schools have the greatest organizational complexity. 

Phase Two: Analysis of Predictors of Academic Salary 

Multiple linear regression is the standard approach for determining the presence or absence of 
gender-based salary differences (Scott 1977; Allard 1984: Johnson. Riggs and Downey 1987: Geetter 
1988: Moore 1993). Single equation regressions are used to investigate the individual and 
organizational sources of faculty salary differences because the effect of each independent variable is 
estimated with controls for the other variables. Multiple R-square and the unstandardized beta 
weights associated with the independent variables estimate the total salary variance explained by the 
model and the average contribution of each variable to annual salary. The unstandardized regression 
weight associated with the sex variable estimates the average gender-based salary inequity. 
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The three organizational factors along with three interaction terms are included in the 
regression equation to assess the organizational effects plus any secondary effects associated with the 
structural factors and female salaries. The magnitudes of the beta-weights and significance levels of 
the interaction terms are examined along with changes in the partial or main effects of the 
organizational structural factors and the sex variable. The R-square and R-square change statistics 
are used to evaluate- the relative contributions of the individual and organizational variables to gender- 
based salary differences. Finally, the organizational effects on salaries are converted to their dollar 
equivalents and the total salary disparity for female faculty is summarized by i^CES institutional 
categories. 

RESULTS 

The study found that the 1987 average salary for the 1,430 female faculty was $36,942 
compared to $47,922 for the 5,106 male faculty. Table 2 shows the impact of each variable group on 
the explained variance in average annual faculty salaries. Over 81% of the variance is accounted for 
by individual faculty variables, such as level of degree earned, academic discipline, initial and current 
rank, and administrative responsibilities. While the organizational variables, size, wealth, and 
complexity, explain 25 percent of the salary differences, their unique contribution to an increase in 
the explained variance (R-square change) is only 2.6 percent. When the organizational interaction 
terms are added to the equation, they increase the explained salary variance by less than 1 percent 
The R-square change estimates the unique contribution of the organizational variables and. thus, is a 
conservative estimate. In addition, since these 19 sample institutions were selected from the same 
public university system, the effects of isomorphism tend to underestimate the organizational 
influences associated with salary differences. 

TABLE 2 

Estimated Regression Effects of the Variable Groups on Gender-Based Salary Differences 









Total 




Gender-Based 








Variance 


R-Square 


Salary 


Variable Groups 


R-Square 


Regressions 


Explained 


Change 


Differences 












-SI 0,980 


1. Individual Faculty Variables 


81.5% 


1. 


81.5% 


81.5% 


-S 1,363 


2. Organizational Factors 


25.1% 


1. + 2. 


84.1% 


2.6% 


-51,241 


3. Organizational x Sex 


1.8% 


1. + 2. + 3. 


84.3% 


.1% 


-S659 



Variables 



The unadjusted male/female average salary difference of almost $11,000 is reduced to only 
$1,363 by the individual faculty variables. Adding the organizational factors after the individual 
characteristics to the regression has a small impact on the magnitude of the gender-based salary 
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difference (-$122). When the interaction terms associated with the organizational characteristics are 
entered, nearly fifty percent of the remaining salary disparities are redistributed (-$1,241 to-$659). 
Thus, for the sample as whole, female faculty received $659 less than comparable males. 

Table 3 presents the regression results for the 37 variables included in the final salary' model. 
The first column of numbers in the table shows the relative dollar influence of each variable on salary 
controlling for all other variables. The most prominent influences on salary are: educational level, 
being in the fields of business and management, medicine, or science and math, holding a senior level 
initial or current rank, being on a 1 1 or 12 month contract, holding an administrative title, and being 
at a well-funded, large, or complex institution. Two variables, age and race/ethnicity were dropped 
from the analysis because they are not significant or introduce problems of collinearity. The 
race/ethnicity category "black, non-hispanic" was not stable during the analyses and not significant in 
the individual faculty characteristics model. Age is moderateiy correlated with salary, however, its 
influence is overshadowed by two other variables in the model, years since highest degree and total 
years of experience. 

Controlling for all the other variables, this study found that the dollar value of being a faculty 
member at an affluent institution (one with a higher level of expenditures per student) in 1987 was 
$2,086. Faculty salaries at larger institutions are, other things being equal, $1,636 higher. The dollar 
value of faculty employment at institutions with greater numbers of vice presidents and deans was 
$1,688 in 1987. Thus, the three organizational factors together accounted for almost $5.41 1 in salary 
variance explained. 

These findings, then, are consistent with other higher education and wage attainment research 
that found male and female faculty alike were paid more because they worked in large rather than 
small organizations and wealthier rather than less affluent organizations. It also provides evidence for 
larger pay associated with employment at more complex organizations. 

As shown in Table 3, there are significant negative effects resulting from the interaction 
between the three organizational factors and gender. This means that female faculty tended to be paid 
$2,710 less than comparable male faculty in the wealthier, larger, and more complex institutions in 
the sample, controlling for the other variables in the analysis. Thus, the data suggest that salary 
disparities are unevenly distributed across higher education institutions. 

To examine further the uneven organizational distribution of gender-based salary differences, 
the salary associated with each factor is estimated separately for males and females in the study. A 
difference factor score between them is calculated. These data are sorted by NCES institution type 
and the salary differences are presented in Table 4. The largest gender-based average salary- disparity 
is observed at medical schools (-$4,652), followed by doctoral institutions with medical schools 
(-$3,033) and lastly, by doctoral institutions (-$2,007). In terms of dollars, there are mildly positive 
effects of >he organizational characteristics on female salaries at the comprehensive ($369), 
baccalaureate ($251), and two-year institutions ($388). Thus, it appears that gender-based salary 
disparities in this particular state university system are not a system-wide "problem", but rather are 
more prominent at the medical and doctoral car :puses. 
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Table 3: Individual, Organizational, and Organizational x Sex Interaction Effects on Salary 



Multiple R « .91856 df « 32, 6503 

R- Squared « .84375 F = 1097.37 



Standard Error = 6269.25 
H = 6,536 



INDIVIDUAL CHARACTERISTICS (a) 
Demographic: 
Sex 

Education: 
less Than Doctorate 
Ed.D. or Ph.D. (b) 
Other Doctorate 

Years Since Highest Degree 

Business t Management 
Health .Related Professions 
Arts I Sciences: Humanities 
Medicine 

Professional Programs 
Science I Mathematics 
Arts I Sciences: Social Sciences 
Two-Year Programs (b) 

Experience: 
9 of 10 Month Contract (b) 
11. or 12 Month Contract 
Other Contract Length 

Initial Rank: Instructor/Lecturer 

Initial Rank: Assistant Professor 

Initial Rank: Associate Professor 

Initial Rank: Professor 

Initial Rank: Leading Professor 



(b) 



beta (dollars) 
-659.08 



-2,098.75 
0 

5,547.45'" 
174.18 

7,838.09 
878.28 

-1136.26 

6,913.05 
611.64 

2,232.06 
822.74 
0 



10,966.60 
-409.31 

0 

33.78 
2,270.47 
7,956.77 
10,468.86 



Rank: Instructor/Lecturer (b) 0 

Rank: Assistant Professor 3,856.67 

Rank: Associate Professor 8,931.51 

Rank: Professor 16,582.69 

Rank: Leading Professor 33,055.38 

Years in Current Rank 69.07 

Department Chairperson 3,401.12 

Other Administrative Title(s) 3,989.62 

Previous Administrative Title(s) 2,798.56 

Total Years of Experience 99. SO 

ORGANIZATIONAL FACTORS 

Organizational Wealth Factor 2,086.01 

Organizational Size Factor 1,636.38 

Organizational Complexity Factor 1,688.14 

Interaction Terms 

Organizational Wealth x Sex -1,438.74 

Organizational Size x Sex -740.55 

Organizational Complexity x Sex -531.01 

(CONSTANT) 22,717.13 



SE.E. 
227.06 

247.77 

396.54 

16.38 

545.97 
520.00 
417.38 
620.95 
428.29 
429.65 
431.53 



383.22 
1151.91 



239.45 
305.97 
379.89 
1489.74 



389.05 
404.90 
451.60 
1037.70 

15.51 

309.39 
360.03 
252.55 

13.92 



145.07 
74.07 
78.42 



241.05 
140.95 
166.67 
562.07 



P 

.0000 

.0000 

.0000 

.0000 

.0000 
.0913 
.0065 
.0000 
.1533 
.0000 
.0566 



.0000 
.7224 



.8878 
.0000 
.0000 
.0000 



.0000 
.0000 
.0000 
.0000 

.0000 

.0000 
.0000 
.0000 

.0000 



.0000 
.0000 
.0000 



.0000 
.0000 
.0014 
.0000 



Variables dropped from the analysis include age and race/ethnicity. 
Reference category. The dollar amount associated with a given variable is in comparison to this group. 



* - Statistical significance 
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TABLE 4 

Estimates of the Organizational and Individual Sources of Sex-Based Salary Differences by NCES 

Category 

Gender-Based Salary Differences From : 



NCES 
Institutional Org. 

Types Wealth 

Medical School S-4,441.59 

Doctoral & -472.31 
Medical 



Org. Org. 

Size Complexity 

S 438.98 S 0.70 

-1,861.62 -48.54 



Total 

Organizational Partial 

Characteristics Effects 

S-4,001.90 S-650.37 

-2,382.47 -650.37 



Individual and 
Organizational Sources 
for Female Faculty 

S-4,652.28 

-3,032.84 



Doctoral 114.05 -421.30 -1,048.97 -1,356.22 -650.37 -2,006.59 

Comprehensive 636.87 59.59 322.60 1,019.06 -650.37 368.69 

Baccalaureate 432.49 621.05 -152.31 901.24 -650.37 250.87 

Two-Year 788.51 366.07 -116.36 1,038.22 -650.37 387.85 
Conclusions and Implications 



The results of this study suggest that the salaries of female faculty at the smaller, less wealthy 
and less complex organizations are within a few hundred dollars of what might be predicted from 
their individual levels of education, rank, and academic discipline. In fact, the historic salary 
disparity between male and female faculty at many two-year and four-year institutions appears to 
have largely corrected itself in favor of women. On the other hand, larger, wealthier and more 
complex doctoral organizations not only pay higher salaries but continue to have greater salary 
disparities between male and female faculty. 

The results of this research are consistent with the wage attainment and salary equity research 
suggesting that salary allocation decisions may be different for males and females in larger, more 
affluent institutions (Cohn 1971; Astin and Bayer 1972; Cox and Astin 1977; Stolzenberg 1978; 
Baron and Bielby 1984; Hodson 1984; and Tolbert 1986). While larger and wealthier institutions pay 
both males and females higher salaries, males appear to benefit considerably more from working in 
these organizations than do females. In addition, this research finds the same effects for the measure 
of organizational complexity; that is, the complexity of an institution increases the salaries of all 
faculty but males benefit more than females from employment at these institutions. 

Without controlling for relevant individual and professional characteristics, Tolbert (1986) 
found unequal distribution of male/female salary^differences across academic markets, and concluded 
that employer preferences may be easier to sustain in organizations that are better able to dominate 
their environments and that preferences may be not as easily sustained in less dominant organizations. 
The curre/ii study produced a similar result after controlling for an array of individual characteristics, 
and thus gives increased support to Tolberfs 1986 conclusions. 




The demographic, education, work experience and organizational variables in our salary 
model explain 84 percent of the annual salary differences (FY87), providing a comprehensive basis 
for comparing faculty salaries. After controlling for the faculty credentials in the model, this study | 
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finds that comparable men and women are differentially compensated for being employed by the 
more dominant doctoral granting institutions. 



This study utilizes a more narrow range of institutions than some of the earlier organizational 
studies. For example, Cohn (1971) and Tolbert (1986) both used national samples of higher 
education organizations and Baron and Bielby (1984) and Hodson (1984) were concerned with 
economic segmentation across industries in California and Wisconsin, respectively. In these cases, 
greater variability in organizational size affected salary levels to a larger extent than in our analysis 
which finds stronger salary influences from organizational expenditures per student than from size. 
It is perhaps more important to observe that the organizational variables as a group exert a $5,41 1 
influence on academic salaries. We believe these affects, however, are understated, not only due to 
the influence of organizational isomorphism, but also due to the HEGIS discipline classification. 
Some of the HEGIS categories, such as medicine and two-year college technologies, capture 
considerable variance associated with the organizational measures. 

Our findings are consistent with those of Scott and Bereman (1992) regarding the enomous 
influence of the academic discipline on salary. Faculty in business, medical, and scientific fields 
receive substantially more salary than their counterparts. While there have been striking changes in 
the gender composition of the academic workforce since the 1970s, female faculty in greater numbers 
than males are still segregated into disciplines and institutions that pay less well (Ransom 1990). The 
unequal distribution of salaries among academic occupations suggests that higher education 
institutions have rather well developed, independent systems of reward and supports the notion of 
firm internal labor markets postulated by Doeringer and Piore (1971). An important extension of this 
research would be to examine credential distributions and dollar returns to specific credentials by 
gender and type of academic market. Such an analysis may suggest the operation of dual labor 
markets, one for males and another for females, especially within the doctoral university markets as 
distinct from the less dominant, four-year and two-year college markets. 

Thus, the current study finds ample support for existing structural/functional approaches and 
explanations for male/female academic salary differences. Congruent with the study by Smart 
(1991), our findings also support a human capital perspective. The human capital approach has 
viewed differences in personal investment as the primary reason for females earning lower salaries 
than males. Career and family choices that influence personal investments in education, professional 
training, and career mobility produce legitimate salary differences between male and female faculty. 
Indeed, our study finds significant and large differences in average male/female salaries to be 
explained by females attaining less education, fewer years of experience, lower ranks and initial 
ranks, and fewer administrative responsibilities. 

We find that the credentials of women faculty as a group disadvantage their annual salaries to 
a much greater extent than salary differences from potential inequities. On the other hand, our 
research substantiates earlier findings suggesting that when university women make the same career 
choices as comparable men. they still do not make as much money, controlling for the variables in the 
model (Astin and Bayer 1972; Katz 1973; Darland et al. 1973; Bayer and Astin 1975: Ferber and 
Green 1984: Barbezat 1987). Thus, male/female differences in human capital investments may be 
influenced by perceived differences in salary and promotional opportunities, as well as by personal 
preferences and interests. 

The size and complexity of academic organizations are very closely related, and we found 
that two of our complexity measures (highest degree offered and number of department chairs) loaded 
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more heavily on organizational size and wealth than on complexity. This left the current study with a 
single measure of complexity. If the effects of complexity are of special interest to future 
researchers, then alternative measures should be tried, such as diversity of mission, the number of 
programs offered by award level, and organizational complexity at higher hierarchial levels. 

While this investigation was not able to obtain information on other salary relevant varaibles 
(such as differences in scholarly productivity and teaching effectiveness), we believe that academic 
rank and initial rank approximate the salary effects of these variables. Prior research suggests that 
rank is highly correlated with productivity, experience and education, thus, both salary and promotion 
are estimated by the same determinants (Astin and Bayer 1972; Katz 1973; Scott 1977; Ferber and 
Green 1982; and Sma1 1991). 

Earlier research identified other organizational determinants as important to male/female 
wage differentials, and future research should take these into account. These measures include: per 
capita income of the institution's state, type of control, student/faculty ratios, rank segregation, sex 
composition of the discipline, percentage of Ph.D's employed in each discipline, institutional mission, 
and student selectivity. An investigation of the assumptions underlying faculty career and life-cycle 
choices is another natural extension of this research. Since labor market and non-labor market 
choices of both males and females have been influenced by the economic and demographic changes 
during the last twenty years, subsequent analysis should examine the barriers and opportunities that 
encourage and discourage faculty' from full participation in educational degree attainment, pre- 
employment training, and career advancement. 

One of the difficult policy issues raised by this research suggests that academic women 
earning the largest current salaries may deserve the largest salary adjustments. A S659 across-the- 
board increase for all women may be justified by this general academic salary analysis, but such 
action would do little to correct inequity in the medical and doctoral universities, and it may create 
further inequity for males in the two-year and four-year colleges. Yet, it may be politically difficult to 
correct disparities of several thousands of dollars for the smaller number of academic females at the 
medical and doctoral institutions, while leaving the salaries for the majority of academic women 
unchanged. 

Knowledge that organizational contexts influence salary decisions and that the higher 
rewards associated with the institution of employment are not equally distributed among male and 
female faculty extends our understanding of the effects of university salary practices. To avoid 
perpetuating the salary differences identified by this research, continued assessment of the context 
and consequences of our everyday salary decisions must occur at the institutional, state, and national 
levels. 



240 a 79 

ERIC 



Appendix A: HEGIS Discipline Categories Used in This Study 



Business and Management includes: 
Health Related Professions include: 

Arts & Sciences: Humanities include: 

Medicine includes: 
Professional Programs include: 



Science and Mathematics include: 
Arts & Sciences: Social Sciences: 



Two-year Programs include: 



Business and Management 

Health Related Professions 

Nursing 

Dental 

Optometry 

Pharmacy 

Fine and Applied Arts 
Foreign Languages 
Letters 

Medicine 

Agriculture and Natural Resources 

Architecture and Environmental Design 

Education 

Home Economics 

Library Science 

Public Affairs and Sen ices 

EOC Programs 

Biological Sciences 

Computer Information Sciences 

Engineering 

Mathematics 

Physical Sciences 

Area Studies 

Communications 

Psychology 

Social Sciences 

Interdisciplinary Studies 

Law 

Business/Commerce Technology 
Data Processing Technology 
Health Sciences/Paramedical Tech. 
Mechanical/Engineering Technology 
Natural Science Technology 
Public Service Technology 
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Influence of Background Variables on Students' Evaluations of Faculty 

Stuart L. Rich 
Director of Institutional Research 
Georgetown University 

In a "Point of View" column in The Chronicle of Higher Education (July 21, 1993), Peter 
Seldin wrote: 

One reason that student evaluations of teachers have become so popular is that they 
are easy to administer and to score. But they also are easy to abuse. If they are to 
shed meaningful light on teachers' performance, the ratings must be used in a way 
that reflects at least some of what we've learned about them from the research 
literature and from experience. . . . Despite the fact that hundreds of studies have 
determined that student ratings generally are both reliable (yielding similar results 
consistently) and valid (measuring what the instrument is supposed to measure), some 
faculty members and administrators argue that factors beyond professors' control bias 
the ratings. 

The literature on students' evaluations of faculty yields conflicting evidence on the influence 
of background variables— course level, class size, discipline, grades, sex and rank of faculty, for 
example— on their ratings. Some studies report no relationship between class size and ratings; others 
suggest a modest relationship (Aleamoni). Studies that find a relationship between grades and ratings 
generally find low correlations (Gleason), although one study (Kendler) found a strong negative 
correlation (-.75) in a large calculus course. Ratings have been found to be slightly higher in the 
humanities than in the social and natural sciences (Centra). Age of die instructor has been found to 
have a small effect on the students' ratings, with older faculty receiving lower ratings (Cashin). Sex 
has been found to have a small effect, but the differences generally have neither statistical nor 
practical significance. Regular faculty tend to receive higher ratings than graduate teaching assistants 
or other less permanent faculty (Braskamp, et. aL). Faculty continue to question the validity of the 
rating instrument— especially if their ratings are not as good as their colleagues'. In Peter Seldin's 
words, "The monumental investment of self that teaching demands predisposes professors to be 
sensitive to criticism of their performance." Because there is such a variety of findings in the 
literature, practically any faculty member with a complaint about her/his ratings can find a study 
which can be used to explain why his/her ratings are low or why the colleague's ratings are 
inappropriately high. If such challenges arise in a particular institution, it behooves the institutional 
researcher to be prepared to resolve the issues for that institution and its particular set of 
circumstances, which may or may not conform to the research results that a faculty member has 
chosen to make his case. In this paper we will describe the methodology, problems, and results which 
were used at one institution to respond to faculty inquiries about the effects of these background 
variables on their ratings by students. 

Our data base consisted of some 3,500 undergraduate courses taught during academic years 
1990-91 and 1991-92. Because our primary interest was in undergraduate teaching, we eliminated 
courses open primarily to graduate students, although graduate students may enroll in some of the 
courses which we did include. We matched the course records with a faculty data base in order to 
obtain sex, rank, and age of the faculty member who taught each course. Because the faculty /course 
rating instruments are completed anonymously, it is not possible to match each student's grade or 
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demographic characteristics with his/her evaluation responses. We therefore calculated an average 
grade for each course section for comparison with the average evaluation score for the course. The 
evaluation instrument used at Georgetown is basically "summative", although there is a special section 
of the form which invites students* written comments, which may be more useful to the faculty 
member for "formative" purposes. Student evaluations are used extensively, but not alone, for 
faculty personnel decisions at Georgetown. 

Both students and faculty tend to focus on the "overall" question— "What is your overall 
evaluation of the instructor?" So our first step was to examine Pearson correlation coefficients of the 
"overall" question with each of the other questions. As indicated in Table 7, the overall question's 
correlations with the other questions range from approximately .20 to as high as .92. The 
correlations with those questions which deal with the instructor (QUI. 1— QUI. 4) are "stro~ v ', i.e., 
from .70 to .92. 

Table 1. 

Correlation of QIIL5. (Overall) with:* 





EL 


Course 










HI. Instructor 








QH.l 


QEL2 


QIL3 


Qn.4 


QEL5 


QIL6 


Qffl.1 


Qm.2 


Qm.3 


Qm.4 


Business Admin 


0.8892 


0.7611 


0.7844 


0.7806 


0.4650 


0.8822 


0.8870 


0.9208 


0.8701 


0.9168 


Humanities 


0.7728 


0.6246 


0.5237 


0.4180 


0.2311 


0.7762 


0.7684 


0.8882 


0.6978 


0.8813 


Math/Science 


0.7121 


0.5561 


0.3734 


0.2726 


0.2865 


0.7460 


0.4515 


0.6283 


0.7936 


0.8867 


Social Sciences 


0.8276 


0.6098 


0.3976 


0.2571 


0.1938 


0.8344 


0.7944 


0.8526 


0.6837 


0.8959 


TOTAL 


0.7921 


0.6249 


0.4929 


0.3779 


0.2258 


0.7954 


0.7328 


0.8478 


0.7250 


0.8900 



Note: All correlations are significant at the .01 level. 
*EL Course Items 

1. To what degree were the stated objectives of the course met? 

2. How effective were the readings, research, and other requirements in helping you to meet these 

objectives? 

3. Did the exams or other graded material fairly represent the content and skills taught in the course? 

4. Do you believe that exams and other work were graded fairly? 

5. To what degree do specialized forms of study (labs, language drills, field trips, e.g.) contribute to the 

value of the course? 

6. How much have you learned in the course? (Your assessment of the quality and extent of that learning) 

*HL Instructor Items 

1 . Does the instructor seem consistently well prepared for class? 

2. Is the quality of classroom presentation stimulating (consider effectiveness of discussions, demonstrations 

and lectures)? 

3. How available, willing and helpful is the instructor in advising and assisting students outside of class? 

4. Does the instructor es'ablish high standards, challenge you, and encourage you to do your best work? 

5. What is your overall evaluation of the instructor? 



Based on the strength of the relationships of the "Instructor" questions with the overall 
question, it was decided to use the overall question alone as the dependent variable in the analysis 
which follows. While the rating scale for all questions on the evaluation instrument ranges from 1 
to 5, actual evaluation scores range from about 3.5 to 5.00; a rating of 1.56 was the lowest received 
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in any course during this period and the 5th percentile was about 3.30. The distribution of responses 
is displayed in Table 2. One of the few background variables about which the research literature 
seems to be consistent is the "discipline". We grouped the departments into the four disciplines 
shown in the tables, because our departments vary greatly in size— from as small as three members 
to over 30. The distributions shown in Table 2 show the discrepancy between the humanities and 
social and natural sciences which we would expect. 

Table 2. 



Distribution of Faculty Evaluation Scores (QIIL5.) 





No. of 










Percentiles 














Courses 


Min 


5th 


10th 


25th 


50th 


75th 


90th 


95th 


Max 


Mean 


Std Dev 


Business Admin 


290 


1.93 


3.16 


3.48 


3.98 


4.33 


4.62 


4.83 


4.91 


5.00 


4.2103 


0.5737 


Humanities 


1,856 


2.00 


3.38 


3.67 


4.11 


4.50 


4.75 


4.91 


5.00 


5.00 


4.3720 


0.5059 


Math/Science 


322 


1.56 


3.22 


3.47 


3.94 


4.31 


4.63 


4.80 


5.00 


5.00 


4.2139 


0.5697 


Social Sciences 


1,036 


2.00 


3.30 


3.56 


4.00 


4.38 


4.63 


4.83 


4.93 


5.00 


4.2690 


0.5026 


TOTAL 


3,504 


1.56 


3.30 


3.61 


4.03 


4.41 


4.69 


4.88 


5.00 


5.00 


4.3136 


0.5208 



Using the SAS regression procedure, we examined the effect of course level, course size, average 
grade, faculty status (regular/adjunct), sex, age, and discipline on the average overall evaluation score 
received in each of the 3,500 courses. The results of two different approaches (five regression models) 
are contained in Table 5. In the first approach we entered discipline as a dummy (1/0) variable; in the 
second approach, we examined the effect of the explanatory variables within each discipline separately. 
The "R-square" statistic may be thought of as the proportion of variation in the dependent variable which 
the model explains. The R-square for Model 1 is .0896, i.e., it explains about nine percent of the 
variation in the average faculty evaluation score. This is reassuring, since the evaluations should truly 
reflect the faculty member's performance and not the background variables which we set out to examine. 
The R-squares for the Humanities and the Social Sciences are similarly less than . 10, although those for 
Business and Math/Science (the smallest of the groups) are about .20. It is typical that the R-square for 
a smaller group would be larger, since there will be less variance among fewer observations. 

We will examine each of the explanatory variables in turn to determine which of them account 
for the explanatory power of the models. The regression coefficients in Table 3 identify the effect of the 
explanatory variable when all other variables are accounted for. For the variables which designate 
"classes" or "categories" (course level, class size, sex, and faculty status), we designated membership 
in one of the classes by creation of "dummy" variables. In the case of "course level", for instance, 
courses at the 200-349 level are given a value of "1" and all others a value of "0". A second dummy 
variable similarly designates courses at the 350^99 level. Courses at the 001-199 level thus become the 
"omitted" class, to which the other two classes are compared. The 001-199 level therefore has a 
regression coefficient of 0.0000, because it has been "omitted". The 200-349 dummy measures the 
characteristics of this level which distinguish it from the 001-199 level. The 350-499 dummy measures 
the difference between evaluations at this level from those at the 001-199 level. 

Course Level. We first examined course level as a continuous variable comprised of course 
number and the square of course number in order to determine if there is a curvilinear relationship 
between course number and the evaluation scores. Neither course number nor its square proved to be 
statistically significant in that model, so we categorized course level into three groups. Courses numbered 
001-199 are primarily introductory and lower division courses, those numbered 200-349 are primarily 
upper division courses, and those numbered 350-499 are mixed upper division and graduate. We had 
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assumed that course level would be collinear with student motivation as measured by choice of major and 
progression into upper division courses in the major, so it came as somewhat of a surprise that m two 
instances the courses numbered 200-349 had a negative effect on the evaluation scores. In Business that 
negative effect is substantial and significant. The "partial R-square" shows the • Proportion of to .tot* 
variation in ratings that individual variables explain. In Business, course level s partial ^quare is 
0316 or about three percent of the variation. In the Social Sciences, courses numbered 200-349 have 
a significant positive effect on evaluation scores. The large positive effect in Math/Science probably 
reflects the collinearity of course level with major, since the introductory courses which are rated 
substantially lower are taken by non-science majors who have a requirement to fulfill. 

Course Size. Runs of the models which included course size (and its square) as a continuous 
variable did not find it to be a significant predictor of the evaluation scores. As a categorical (dummy) 
variable class size showed a significant negative effect for courses which were medium sized (21 -5U). 
The effect of "large" course size was mixed-overall and in the Social Sciences large classes have a 
positive but not significant effect on ratings. This may be due to the predominant use of star faculty 
in the largest lecture classes. Overall, the effect of class size tends to confirm the given wisdom that 
smaller classes are better than large classes. It should be noted that in no case does class size account 
for more than one percent of the variation in the ratings. 

Average Grade. Grades had a significant positive effect on the ratings in each of the models we 
ran Their effect ranged from a low of three percent in the Social Sciences to a high ten percent in 
Math/Science. From earlier studies, we know that grades awarded in the Humanities are significant y 
higher than those awarded in the other disciplines, but since grades in the Social Sciences are typically 
higher than those in Business Administration, it was a surprise to see that the effect of grades in Business 
was actually a bit larger than in the Social Sciences. It should be noted that students evaluate faculty and 
courses prior to final examinations when they do not know with, any certainty what their final grade will 
be. The partial R-square shows that grades generally account for about half of the total R-square. 

Full-time Faculty. With only one exception, full-time faculty receive significantly and 
substantially higher ratings than adjunct faculty. Even though the coefficient is not significant, it is 
interesting to note that adjunct faculty in Math/Science received higher ratings than full-time faculty. The 
largest effect is in Business, where full-time status explains about 2.5 percent of the variation in ratings. 

Sex Women received significantly and substantially lower ratings than men in Business, 
accounting for about three percent of the total variation within the discipline. In the Humanities and 
Social Sciences, their ratings were higher, but by an insubstantial and insignificant amount. In 
Math/Science women's ratings were lower than men's, although their small numbers account for the tact 
that the difference is not statistically significant. 

Age In examining age, we used a variation on the methodology used by Kinney and Smith 
(1989) who hypothesize that age and teaching effectiveness may have a linear/cumulative relationship 
that can be accounted for by age, followed by a decline which can be detected by squaring age. Our 
analysis showed age to have a curvilinear relationship with students' evaluations, generally positive in 
the early years, followed by slow growth or a plateau in the middle years and a rather sharp decline in 
the later years. These relationships are displayed graphically in the plots in the appendix to this paper. 
In those plots we have accounted only for die faculty member's age and for course level and class size. 
It should be noted that the net effect of age is rather small. Age accounts for about seven percent of the 
variation in ratings in Business and about four percent in Math/Science. One cannot help wondering if 
it is possible to determine whether the relationships between students' evaluations and age is truly due 




to declining effectiveness or is attributable to "distance" perceived between the students and their teachers. 
In the early years of a faculty member's career, he/she may be relatively close in age to the students; 
at mid-career he/she will be closer to the students' parents' ages; and in late career more like the 
students' grandparents. 

Discipline. The "all discipline" model, which treated the four disciplines as dummy variables, 
found a substantial positive difference between the Humanities and Business and a smaller significant 
difference between Math/Science and Business, although both accounted for less than one percent of the 
total variation. The difference in ratings for the Social Sciences was neither practically nor statistically 
significant. 

In a stepwise regression for all disciplines, seven of the twelve variables we considered met the 
. 15 significance level for entry into the model. They are listed below in the order of their entry and their 
contribution to the R-square of the model . The total R-square for the stepwise model was .0889, whereas 
the R-square of the twelve-variable model was .0896. 



Variable 


Partial 


Model 


Entered 


R-square 


R-Square 


Grade 


0.0549 


0.0549 


Age 2 


0.0157 


0.0706 


Humanities Discipline 


0.0068 


0.0773 


Faculty Full-time 


0.0074 


0.0847 


Class Size 21-50 


0.0020 


0.0867 


Course Level 350-499 


0.0014 


0.0881 


Math/Science Disc. 


0.0008 


0.0889 



— — j ""-K ui uic vaiiauun ui.Muuenii ratings or iacuuy. uver halt 

of that is accounted for by grades alone. When the disciplines are treated separately (Models 2-5), the 
most variation that these variables account for is twenty percent. These results are within the range of 
variation found in other research reported in the literature. While the overall effect of these background 
variables is small, their influence on the evaluations within disciplines varies. The size of the effects in 
some circumstances is large enough to warrant consideration when assessing the ratings of individual 
courses/faculty who possess the characteristics investigated here. Where students' ratings are used in 
evaluating faculty's teaching performance, it is essential to understand whether factors which are beyond 
the professor's control bias the ratings. The process we have used here is relatively easy for the 
institutional researcher to implement in order to be prepared to answer inquiries from faculty, department 
chairs, and deans who may wish to know to what extent the ratings reflect extraneous characteristics of 
the classes which have been evaluated. 
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An Analysis of Patterns of Federal Support for 
Academic Science and Engineering: 1981-1991 

James F. Trainer 
Director 

The Higher Education Data Sharing Consortium 
Franklin and Marshall College 

Introduction 

This paper provides a longitudinal analysis of the availability of federal funds 
for academic science and engineering for the period 1981 through 1991. It 
examines the allocation of funds to institutions over time and explores shifts in 
the sources of funds vis-a-vis various federal agencies. Particular attention is 
paid to those agencies which appear to serve as potential "gateways" to federal 
funds for institutions that traditionally do not rank at or near the top of the list 
of total federal research dollars received. Funding data are adjusted against 
inflation through the use of the Research and Development Price Index (R&DPI), 
developed by the Research Associates of Washington. These adjustments allow 
for the measurement of any real change in research funding over time. 

In addition to providing trend analyses of information on federally funded 
science and engineering, the paper serves as an example of the type of research 
that can be conducted employing the federally sponsored Computer Aided 
Science Policy Analysis and Research (CASPAR) database developed for the 
National Science Foundation (NSF) by the Quantum Research Corporation of 
Maryland. In this sense, the paper serves as an introduction to the use of federal 
databases for individuals not currently familiar with the availability and use of 
this type of information. 

Background 

Over the past fifty years, the federal government has played an increasingly 
important role in supporting academic science and engineering through the 
allocation of funds for the construction and maintenance of scientific facilities 
and the support of basic and applied research and development activities. While 
the history of federal support for agricultural research dates back nearly a half a 
century prior to 1940, the dawn of World War II ushered in the government's 
first true commitment to academic science. With grants in support of the war 
effort, the government began its role as a major funding agent of science and 
engineering. This support continued through the cold war and into the present 
era. Federal research funds have spawned discoveries in medicine, atomic 
energy, aeronautics, radar, and biotechnology, to name but a few areas. Indeed, 
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the public, as represented by the government, has grown to accept its role in 
supporting academic research (Geiger, 1986 and 1993). 

The allocation of federal funds has contributed to the development of an 
academic research economy. Institutions attempt to attract research funds for a 
variety of reasons. For instance, the availability of dollars to support research 
may be viewed by some institutions as a means to offset declining revenues in 
other areas. Some institutions may see research dollars as a way to decrease their 
tuition dependency. Other institutions may increase their research activity and, 
in turn, their research grant writing in an effort to move up in the oft-maligned, 
yet closely watched, academic prestige rankings, such as those produced by the 
National Research Council or those published by the US News and World 
Report and various other entities. Institutions also may see an increase in 
research activity as they hire younger faculty who have been socialized toward 
grant writing in graduate school and are eager to embark on their research 
careers. Thus, given the potential increased focus on research revenues, there 
were four objectives in doing this study: 

Objectives 

1. To analyze longitudinal patterns of federal support for academic science and 
engineering. 

2. To identify agencies which may provide access to federal funds for 
institutions that traditionally are not among the top institutions in terms of 
total federal dollars received. 

3. To examine the real availability of federal science and engineering funds 
relative to various measures of inflation. 

4. To demonstrate the types of research that can be conducted using the 
CASPAR database and other publicly accessible information. 

Sample 

The analyses generally include data on 154 institutions, although some 
information is reported relative to total national research funding and fund 
recipients. The specific sample of 154 includes: The top 100 institutions in 1991 
in terms of federal obligations for science and engineering, and 54 additional 
selected institutions. Data are sorted and analyzed for 14 different federal 
agencies and 8 academic disciplines. 
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Data Sources 



1. NSF Survey of Federal Support to Universities, Colleges and Selected Non- 
Profit Institutions (filed annually by federal agencies). Data are employed in 
the study from the surveys from FY1981 through FY1991.* 

2. NSF Survey of Scientific and Engineering Expenditures at Universities and 
Colleges (filed annually by academic institutions). Data are used in this study 
from the FY1991 survey.** 

3. Inflation Measures for Schools and Colleges, published annually by Research 
Associates of Washington. 

"These data were extracted from the Computer Aided Science Policy Analysis and Research 
Database System (CASPAR) developed by the Quantum Research Corporation of Bethesda, 
Maryland. 

Procedures 

Four basic steps were followed in completing this study. First, the appropriate 
data were extracted from the NSF Survey of Federal Support to Universities, 
Colleges and Selected Non-Profit Institutions on the CASPAR database for each 
one of the 154 institutions in the study's pool. The extracted data included the 
total dollars awarded by each of the 14 agencies nationally as well as the dollars 
awarded specifically to each of the 154 institutions in this study from the years 
1981-1991. Data were also pulled from the NSF Survey of Scientific and 
Engineering Expenditures at Universities and Colleges to analyze the 1991 R&D 
expenditures at each of the 154 institutions by source of funds and by science and 
engineering disciplines. A count was also made of the total number of 
institutions that received funds from each of the 14 agencies in each year of the 
period studied. 

Once the raw data were extracted from the database, all dollar amounts from 
previous years were converted into 1991 dollars. These adjustments were made 
by multiplying a given year's actual dollars by a factor which accounted lor any 
inflationary growth between the year in question and 1991. The R&DPI was 
employed in making these adjustments, in that it best reflects changes in the 
costs of doing research. As an example, 1981 actual dollars were multiplied by a 
factor of 1.7 to convert them into 1991 constant dollars. Similarly, 1990 dollars 
were converted by a factor of 1.05. 

Step three of the study involved measuring both the distribution of total 
available federal funds across agencies and institutions, and shifts in funding 
patterns over time. The effort focused on measuring any real growth and shifts 
in funding proportions between 1981 and 1991. Both percent distributions and 
percent changes were calculated. 
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The final step of the study centered around developing frequency counts of the 
number of years in which each institution received any funds from each of the 
studied agencies. These counts afforded a measure of any trend which may have 
developed regarding both the exclusivity of grant making by any agency and the 
dependency of any given institution on any one source or combination of federal 
funds. The analysis of these trends is helpful in identifying agencies which serve 
as "gateways" to federal funding. 

Results 

In Fiscal Year 1991 the 154 institutions examined in this study received 59% of 
their science and engineering research funds from the federal government. 
Institutional funds accounted for another 18% of their overall research revenues 
and expenditures. The remaining 24% of their research revenues were received 
in equal proportions from state and/or local governments, industry, and other 
sources (Figure 1). 

These 154 institutions spent 50% of their research funds in the life sciences. 
Another 15% of their funds were expended in engineering, and 13% were spent 
in the physical sciences. Geosciences received 7% of available research funds and 
math/ computer science and the social sciences each received 5%. The remaining 
5% of the institutions' research budgets were split between psychology and other 
scientific disciplines (Figure 2). 

Total federal funding for academic science and engineering activities grew 
37.7% over and above inflation in the period from 1981 through 1991. The 
greatest percentage gains in available funds occurred in the Department of 
Education, the Department of Health and Human Services, and NASA, with 
growth rates of 261%, 197% and 90.1% respectively. Only the Nuclear Regulatory 
Commission (-57.3%), other agencies (-52.9%), and the Department of 
Transportation (-7.6) lost ground against inflation in terms of the research funds 
they awarded at the beginning and end of this period. 
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Figure 1 

R&D Mean Expenditures by Source, 1991 
Selected Universites 



State/Local 7% 




Federal 59% 



Figure 2 

R & D Mean Expenditures by Field, 1991 
Selected Universities 



Psychology 3% 

Inter/Other 2% 

Social Science 5% 



Math/Computer 5% 



Geo Sciences 7% 



Physical Science 13% 




Engineering 15% 
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Life Sciences 50% 



Table 1 



Federal Obligations for Science and Engineering in Constant 1991 Dollars 





1981 


1986 


1991 


1981-91 


Federal Agency 


($000) 


($000) 


($000) 


A 


Agriculture 


835,397 


794,490 


867,424 


+ 3.8% 


Commerce 


88,070 


89,948 


121,910 


+ 38.4 


Defense 


1,234,253 


1,543,900 


1,539,231 


+ 24.7 


Education 


95,328 


94,721 


344,157 


+261.0 


Energy 


527,660 


538,287 


658,905 


+ 24.9 


EPA 


114,794 


90,565 


115,925 


+ 1.0 


Interior 


58,184 


52,301 


80,390 


+ 38.2 


NASA 


311,950 


345,984 


592,923 


+ 90.1 


NIH 


3,666,885 


4,190,948 


4,968,252 


+ 35.5 


NSF 


1,181,677 


1,351,450 


1,642,261 


+ 39.0 


NRC 


12,575 


4,826 


5,366 


- 57.3 


Other HHS 


351,252 


335,080 


795,148 


+197.7 


Transportation 


48,521 


17,356 


44,838 


- 7.6 


Other 


80,310 


61,661 


37,809 


- 52.9 


Total Federal 


8,606,855 


9,511,516 


11,814,539 


+ 37.3 



As Table 1 shows, the vast majority of federal funds for academic science and 
engineering are available through a relatively few federal agencies. The 
Department of Defense, the National Institutes of Health, and the National 
Science Foundation alone combined to account for over two-thirds of the nearly 
12 billion dollars worth of federal grants made in science and engineering in 
fiscal year 1991. These agencies accounted for approximately the same proportion 
of total grant dollars in 1981. These three agencies also lead the way in granting 
money to the highest number of institutions. In 1991, for instance, of the total of 
1141 institutions that received federal grants in science and engineering, 735 
received funds from the NSF, 432 received funds from NIH, and 321 were 
awarded funds from the Department of Defense. However, each of these 
agencies also contribute at least 25% of their funds to ten institutions. In fact, of 
the 321 institutions that received funds from the Defense Department, ten 
institutions combined to received 52.7% of all of the funds allocated by this 
agency. All three of these agencies, however, allocated a smaller proportion of 
their funds to the top ten institutions in 1991 than they did in 1981; 
demonstrating at least the appearance of a desire to distribute more dollars to a 
greater number of institutions. 
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Table 2 



Number of Institutions Receiving Federal Grants for Science and Engineering 
and the Percent of Total Dollars Awarded to each Agency's Top Ten Recipients 







1981 






1991 




Agency Sample N 


National 


Top 10 


Sample N 


National 


Top 10 


Agriculture 


85 


189 


30.4% 


88 


232 


28.3% 


Commerce 


67 


116 


39.8 


73 


121 


39.5 


Defense 


122 


274 


61.3 


132 


321 


52.7 


Education 


74 


132 


38.8 


129 


660 


21.0 


Energy 

CD J 


115 


240 


41.8 


124 


279 


34.1 


FPA 




186 


25.8 


110 


236 


28.0 


Interior 


78 


158 


24.1 


75 


176 


29.3 


NASA 


122 


271 


34.4 


126 


319 


36.5 


NIH 


136 


406 


26.8 


141 


432 


25.9 


NSF 


146 


661 


30.0 


148 


735 


25.5 


NRC 


32 


45 


64.6 


27 


36 


60.5 


Other HHS 


125 


83* 


31.6 


129 


51* 


31.3 


Transportation 


72 


116 


57.4 


39 


86 


39.9 


Other 


46 


** 


59.7 


29 


** 


66.3 


Total 


152 


899 


24.6 


154 


1141 


20.9 



The national numbers in this category include only those institutions that received funds from other 
HHS entities but not the NIH. The sample numbers include those institutions that received funds from 
other HHS entities as well as possibly the NIH. 

**Count not available. 



Although awarding less dollars than most other agencies, the Department of 
Education supplied funds to 660 institutions in 1991, second only to the NSF in 
the number of grants awarded. Unfortunately, however, grants from the 
Department of Education accounted for only 2.9% of the total federal research 
dollars allocated for science and engineering in 1991 (Figure 3). Unlike the more 
lucrative funding agencies, which also tend to grant a large proportion of their 
funds to relatively few institutions, the Department of Education awarded only 
21% of its grant dollars to the top ten of its 660 recipient institutions. No other 
single agency awarded less than 25.5% of its research funds to the top ten 
institutions on its list as evidenced by the data reported in Table 2. 
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Figure 3 

Distribution of Federal Funds by Agency in 1991 



Other (6%)* 




NSF (14%) 



*Other category includes Commerce, Education, EPA, Interior, NRC, 
Transportation, and Other Federal Agencies. 
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The top ten institutions, in terms of total federal dollars received, were 
granted 20.9% of federal funds for science and engineering in 1991. In addition, 
the list of institutions at and near the top of the federal obligations list has 
remained relatively stable over time. These institutions also tend to receive 
funds from a variety of agencies. However, while institutions at the top of the 
list attract nearly all the funds from certain agencies, they tend to attract a smaller 
proportion of funds from other agencies. Thus, those agencies not awarding the 
majority of their funds to the top institutions, in terms of dollars received, may 
serve as "gateways" to federal funds for certain institutions. These agencies 
include the Department of Education and to lesser degrees the N1H and the NSF, 



Conclusion 

Over the last half century a symbiotic relationship has developed between the 
federal government and our nation's colleges and universities. The 
government, for its part, has contributed funds which have fostered the 
development of the academic research economy. Numerous institutions and 
thousands of research scientists have received funds supporting a variety of 
research enterprises and endeavors. In exchange for this support, academic 
scientists have made important discoveries in advancing medical science, our 
national defense, the space program, and many other areas crucial to our 
nation's security and well-being. 

However, changes may be on the horizon in terms of federal support for 
academic science. As the nation wrestles with the federal deficit, one has to 
wonder whether the funding trends of the past will continue. For example, 
some may fear that the congressional cancellation of the super collider project 
may mark the beginning of the end for federally funded "big science." Likewise, 
some institutions may be negatively affected by down-sizing at the Department 
of Defense. Further, all institutions that receive federal research funds should be 
concerned about the potential fallout from the recent scandal related to 
inappropriate indirect cost/overhead charges, and the current public outcry over 
congress' non-competitive "scientific pork barrel" grant making. In short, 
institutions should be concerned that the public's interest in supporting 
academic research is waning. Indeed, if this is the case, we may need either to 
take steps to reaffirm the public's trust in academic research or to re-examine the 
academic research economy's dependency on federal funding; either option is 
bound to have a significant impact on the way we fund, conduct, and promote 
academic research. 
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Introduction 

Campus crime, while receiving heightened media attention the past two to three years, is not 
yet well understood in the academic community. Very few investigative studies have been conducted 
on the correlates of campus crime. If the issue continues to receive heightened media coverage, the 
higher education community must be able to show that it understands the problem and is addressing 
it. Numerous civil court cases have also pointed out the need for campus administrators and 
executives to be cognizant of their duties and responsibilities with respect to protecting students from 
criminal activity (Richmond, 1990). This study examines* the correlates of campus crime. Its focus 
is on the characteristics of campuses, of students, and of the communities that surround them that are 
associated with crime on campus. 

Providing insights to campus crime should help students, faculty, and administrators alike to 
develop more effective responses and strategies to address crime on campus. Such information also 
should increase understanding among trustees, legislators, parents, and others concerned about the 
problem. 

Residential campuses are unique institutions in American society because many have a 
relatively homogeneous population with respect to age, are comprised of a highly mobile population, 
and have a well defined sense of university community. Furthermore, colleges and universities pose 
an "environment that can be subjected to alteration and control" (Fox and Hellman, 1985). 
Additionally, universities have direct responsibility for, if not control over dormitories and other 
campus buildings. Even with the demise of in loco parentis, colleges and universities are generally 
responsible for student safety, especially on campus, and are widely believed to exert considerable 
influence on the personal activities of their students(Richmond, 1990). 

The Student Right-to-Know and Campus Security Act of 1990, largely a result of a grass 
roots movement to make campus crime data available to current and prospective students, has further 
directed public attention toward campus criminal activities and safety. For the first time, in 1993, 
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every college and university receiving federal funds is required to issue, upon request, an annual 
security report to employees, to students and their parents, as well as to the secretary of education. 
The report is to include a statement of security policies, and crime statistics for the preceding three 
years covering the crimes occurring on campus in the following categories: murder, rape, aggravated 
assault, burglary, motor vehicle theft, and alcohol and drug violations. 

The Chronicle of Higher Education recently published college and university crime rates for 
nearly 2,400 institutions of higher education ( Chronic le, Jan. 20, 1993). The Chronicle reported that 
there were 7,500 incidents of violent crime on U.S. campuses in 1992, including 30 murders, 1,000 
rapes, 1,800 robberies. In addition, there were 32,127 burglaries and 8,981 motor vehicle thefts. 

While many concerned individuals and organizations welcome the new law requiring the 
disclosure of campus crime data, many college officials see the disclosure as a potentially damaging 
act. They express a concern that people will use the crime data to "sensationalize or stereotype 
institutions" ( Chronicle , July. 22, 1992). Indeed, the legislation does not provide a context for 
interpreting the data. It does not seek to distinguish between residential and commuting institutions. 
Institutions with very small enrollments may appear to have very low instances of crime vis-a-vis 
large institutions, but when enrollments are taken into account they may in fact have higher per capita 
crime rates. Colleges are weary that crime numbers will be used out of context. Most administrators 
believe that there is significantly less crime on campuses than in communities ( Chronicle , Jan. 20, 
1993), which, as we will discuss later, our study bears out. 

The debate about publishing campus crime continues though. Many people familiar with the 
issue believe that colleges are merely concerned with their images, not about safety. To them the 
debate over crime reporting is about its impact on money generated by enrollments and alumni 
contributions ( Chronicle , July. 22, 1992). Regardless of one's feelings about campus crime reporting 
it is the law, and campus crime is an issue of concern to all who attend and support colleges and 
universities. Moreover, there is a dearth of descriptive and analytical studies. 

Theoretical Framework 

Why are colleges and universities susceptible to crime? The conceptual framework for our 
study derives both from the field of criminal justice and from earlier research. Crime in society 
generally falls within the framework of Routine Activities Theory. According to Cohen and Felson, 
most criminal acts require convergence in space and time of likely offenders, suitable targets, and the 
absence of capable guardians. This theory is based on the "supposition that daily work activities 
separate many people from those they trust and from property they value" (Cohen and Felson, 1979). 
Routine activities bring together people of different backgrounds. Likely offenders are found within 
the surrounding community, if not within the student body itself. If a capable guardian is absent, then 
the probability of crime occurring becomes higher. Colleges and universities by their nature, contain 
suitable targets for offenders - people coming and going at all hours, unattended or poorly secured 
buildings, accessible motor vehicles, and items of high value per unit size, such as stereo equipment 
and desk-top computers. Given the dispersed nature of many campuses, guardians such as campus 
police or students are rarely standing watch over valuables. While offenders may be exogenous to the 
campus or students themselves, suitable targets and the absence of guardians lead to Cohen and 
Felson's convergence that explains the occurrence of campus crime. 

Campus crime occurs, but what influences or contributes to its occurrence? Fox and Hellman 
(1985) conducted a study of factors that influence the total campus crime rate, as reported by the 
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FBI's Uniform Crime Reports. They examined such things as student characteristics, structural 
features of the campus, administrative staffing, and location. This study was published several years 
prior to the recent explosion of media attention on campus crime and used data from 1980. The 
authors' found colleges and universities have less crime than their surrounding communities and 
location had little or no influence on the ratio of campus to community crime. Among the 
correlations between campus crime and university characteristics found by Fox and Hellman were: 

• Positive, significant correlations between campus crime and tuition cost, the percent of male 
students, population density, and campus police staffing levels. 

• The percent of minority students on campus was not a significant correlate of campus crime. 

Within the framework of Routine Activities Theory, and using a larger more elaborate 
database, we sought to revisit the earlier 1980 research, especially in view of recent media attention 
and expressed concern by students and parents. 

Methodology 

Using both longitudinal and cross-sectional databases, and both descriptive and multivariate 
analyses, this study examines the trends and correlates of campus crime. The two primary 
aspects of the study examine the trends in campus crime since 1974, and analyze 1990 cross sectional 
data for relationships between campus crime and college characteristics. Fox and Hellman's study 
utilized 1980 crime statistics and campus data for 222 colleges and universities. This current study 
utilizes a similar but expanded dataset of 400 institutions of higher education using several sources of 
data. 

The Consortium for Higher Education Campus Crime Research (CHECCR) has amassed data 
on a sample of 400 college campuses across the country- These data are being compiled by 
CHECCR to better inform institutions of higher education about the causes and correlates of crime on 
campus. Two kinds of CHECCR analyses can be presented to participating institutions. The first is a 
general report about the causes and correlates of campus crime in general. These analyses can be 
broken down by type of crime and characteristics of the institution. The second type is an individual, 
confidential, proprietary report considering how a particular campus ranks relative to others of similar 
size and type. 

Our research draws upon three merged national databases of federal crime statistics, 
community demographic data, and campus characteristics data, the latter from the College Board 
Survey. 

Database Building 

Two data sets have been constructed using the Federal Bureau of Investigation's (FBI) 
Uniform Crime Report (UCR) data on campus crime. The first is a time series data set that plots 
trends in specific types of crime from 1974 to 1991 for colleges that have reported consistently. 
Because the number of colleges reporting instances of crime to the FBI has increased each year, 
another version of this time series uses data on all colleges reporting each year. This data set allows 
us to view trends in specific types of crime over time. 
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The second FBI data set is a cross section of the 400 colleges reporting to the UCR in 1990. 
Variables in the data set include crime rates for specific types of crime, along with characteristics of 
the campus and the community in which the campus is housed. 

These data sets allow us to examine many questions about the character and correlates of 
specific types of campus crime. For example, what are the trends in campus crime since 1974? How 
do community characteristics affect crime rales on campus? 

While many campus and student body characteristics are included in the 1990 UCR data set, 
these data items were improved and expanded upon by merging it with data from the 1989-90 College 
Board Survey. In addition, we utilized information from the Carnegie Commission, Barron's Guide, 
U.S. News and World Report, the College Board, and the Chronicle of Higher Education. By 
compiling the additional data items we were able to expand the focus of the inquiry to examine a full 
range of college and student characteristics in addition to the community characteristics contained in 
the FBI data. 

Variable Reduction 

Merging the 1990 CHECCR crime data and the 1989 college and student characteristics 
database supplied over 475 separate variables as potential correlates of campus crime. Many 
variables provided redundant information, and several had an inordinate amount of missing data 
across many colleges in the sample. These variables were accordingly excluded from the 
examination. However, the broad scope and plethora of original data items provided more than 
enough information to proceed with the investigation. 

Upon review, the remaining independent variables were worked into three broad categories: 
student, organizational, and community variables. The three groupings appear to fit well into Cohen 
and Felson's Routine Activities Theory. Students or their property comprise accessible targets, the 
organizational characteristics act as a surrogate for capable guardians, and community characteristics 
may provide likely offenders. It should be noted that there is not a clear delineation between each of 
the categories. For instance, some organizational characteristics such as the percent of students in 
residence hails could also be thought of as a student characteristic. 

The framework of dividing the variables into three groups also served to aid in additional 
variable reduction. Variables that violated ordinary least squares error assumptions and caused 
considerable multi-colinearity were able to be replaced with variables that explained similar degrees 
of the data variance but did not violate such assumptions. 

The refined dataset of predictors was whittled down to 68 variables - twenty-four surrogates 
of student characteristics, twenty-four surrogates of organizational characteristics, and twenty 
surrogates of community characteristics. Separate factor analyses (principle components analyses) 
were conducted for each grouping, aiding in the adoption of 20 variables for the final regression 
equations. The factor analyses grouped together those variables that explain similar aspects of the 
variance in campus crime. Variables were selected on the basis of having high factor loadings, a 
large number of cases, and lacking colinearity. 
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Table 1 lists the community variables utilized in the initial model and the number of cases 
containing the information. The variables in bold print were used in the final model equations. 
Where applicable, the factor loadings associated with each variable and its corresponding factor are 
shown. 
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380 Arson Rate/100,000 


.58 








380 Police Force Rate/100,000 


.74 








397 Population 




.90 






394 Persons Age 18-20 




.96 






394 Persons Age 21-24 




.86 






394 Number of Female Headed 




.79 






Households 










394 Percent Below Poverty Level 






.81 




394 Percent Unemployed 






.76 




394 Average Income 






-.76 




397 Average Temperature 








.76 



The community characteristics, when subjected to principal components analysis, factored 
into four separate groupings: crime, population, poverty, and climate. We decided to use violent, 
property, and total crime. Also included in the crime factor was a variable for community urban - non 
urban setting. We selected community population and the percent of the community population 
below the poverty level as the best variables from those factors. 

Finally, average temperature of the area was included in the regression. It has been shown in 
the criminal justice literature that temperature does indeed have a statistical correlation with crime 
rates. The warmer an area is the more likely people are outside doing things and leaving their 
valuables unattended - targets for theft. Additionally, interactions between disagreeing parties are 
more likely to turn confrontational in sweltering heat. 

The organizational characteristics also were subjected to principal components analysis to 
identify those variables that tend to vary together. As shown in Table 2, these data separated along 
four basic lines: size, cost, resource base, and density. 
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Table 2 

Organizational Characteristics 

Resource 

N Variable Size Cost Base Density 



355 


Number Faculty with Ph.D. 


.91 




423 


Number FT Faculty 


.88 




355 


Wealth (tricotimized) 


.86 




346 


Number Library Volumes 


.82 




433 


Total Enrollment 


.77 




433 


Total FT & PT Undergrads 


.72 




390 


Number VPs & Deans 


.66 




361 


Percent Students in College Housing 


-.50 




144 


Avg. Associate Professor Salary 




.90 


144 


Avg. Associate Professor Salary 




.89 


144 


Avg. Pull Professor Salary 




.84 


388 


Instate Tuition Charge 




.82 


432 


Tuition cost 




.82 


354 


Room & Board Cost 




.80 


355 


Percent FT Faculty w/ Ph.D. 




.83 


145 


R&D. Expenditures per Student 




.53 


433 


Public/Private 






346 


Library Holdings per Student 






104 


Endowment per Student 






383 


Campus Police Rate 






372 


Student/Faculty Ratio 






378 


Number of Campus Acres 






372 


Campus Police per Acre 






371 


Students per Campus Acre 







.87 
.65 
.63 
-.55 



.93 
.86 
.82 



Based upon the number of cases and the factor loadings, we selected three size variables for 
the final model: the number of full-time faculty, total enrollment, and the percent of students in 
college housing. The relative size of the residence hall operation could serve as a student body 
characteristic as well as organizational characteristic. 



While tuition cost and room and board cost are very similar, they both were used in the final 
model, especially since tuition is a proxy for public/private, while room and board cost relates to 
organizational and student affluence. Full, associate, and assistant professor salaries, while having 
the highest cost factor loadings were only available for 144 colleges, and were eliminated from 
consideration. 



Library holdings per student and the campus police rate (per 100,000) were utilized from the 
resource base factor because they had more valid cases than endowment per student. The student 
faculty ratio caused colinearity problems in the final regressions and was excluded from the final 
analysis. 
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The number of campus acres and the number of students per campus acre were included in 
the final regression as density indicators. While campus police per acre was considered a possible 
indicator of guardian coverage, it did not fit as well as campus police per capita in subsequent 
regression equations. 

Only six of the student characteristic variables made their way into the final model. Table 3 
below shows them. While eight of the variables in the selectivity factor loaded highly, only the 
percent of freshmen in the top 10% of their high school class was included in the final model. The 
other variables in the selectivity factor, while loading high, nonetheless were eliminated due to 
problems with multicolinearity or missing data. 



Tabic 3 

Student Characteristics 

Fraternity/ Financial 

N V ariable elcctivity Sorority Aid Ethnic Demog raphic 

275 Percent from Top 10% of High School .85 

173 1990 Frosh Attrition -.88 

27 1 Percent from Top 25% of High School .8 1 5 

377 Percent Frosh Appl. Accepted - 8 1 5 
336 Barron's Competitiveness 81 
374 Difficulty of Entrance .8 1 
173 1990 Frosh Retention Rate ^ 
171 1990 Graduation Rate 72 

24 1 Percent Females Sorority 

242 Percent Males in Fraternity 
246 Percent Fraternity/Sorority 
372 Percent on Financial Aid 
372 Percent Freshmen \v7 Fin Aid 
34 1 Percent Minority Students 
376 Percent Undcrgrads - Male 
424 Percent in-State Students 

378 Percent Foreign Students 
4 1 5 Average Age of Undergraduates 
4 1 9 Percent Total Students Commuting 
252 Total Transfers Enrolled 
240 Percent Enrolled Frosh w/ Need 
433 Total FT Graduate Students 
305 Percent Accepted that Enroll 

372 Student/Faculty Ratio 



The percent of students in fraternities or sororities was utilized rather than just the percentage 
of males in fraternities or the percentage of females in sororities, even though all three had very high 
factor loadings. 

The financial aid factor contained two indicators of student need: the percentage of freshmen 
with financial aid, and the percentage of all undergraduates with financial aid. The more 
encompassing indicator of total undergraduate student body need was utilized in this case. 



.93 
.92 
.90 

.91 

.87 

.85 

.57 
-.54 
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The percentage of minority students, the percent male, and the percentage of students from 
in-state each had isolated high loadings, and were kept in the final model to incorporate measures of 
student diversity. 

RESULTS 

Trends in Campus crime 

As a first step, we examined the longitudinal data set for trends in specific types of crime 
over time. These crime data are reported per 100,000 students. The campus-crime rates, therefore, 
would be even lower if the database included faculty, staff, and visitors in the population. For 
example, a single sports event on a Saturday night can attract tens of thousands of visitors to the 
campus, and several CHECCR institutions report that a significant amount of their campus crime is 
associated with such events. 

Chart One compares violent crime on-campus to the national trends and shows that 
campuses are over 10 times safer than the nation in general. Violent crime includes murder, 
assault, rape, and robbery, with assault generally constituting over 75% of the incidents, and robbery 
another 15%. In 1991, for example, there were more than 750 violent crimes per 100,000 people in 
the nation, but only about 64 per 100,000 students on campus. 

Chart One also reflects a 27% decrease since 1974 in violent crime on-campus (from 88 to 
64 per 100,000), while crime was increasing in the nation by 41% (from 460 to 758 per 100,000). 
The data for the individual crimes of homicide, assault, rape, and robbery are each relatively 
consistent with this overall trend - rising for the nation as a whole, but falling on campus. As the 
country becomes more dangerous, campuses are becoming safer. These findings are particularly 
striking when one considers that campuses are full of young people, and these are the most likely to 
become involved in crime, whether as victims or as offenders. 

Chart Two compares property crime on-campus to the national trends. Property crime 
includes larceny, burglary, and vehicle theft. Larceny is the largest component of the overall crime 
rate, and generally accounts for over 80% of campus crime and 55% of crime in the country. 
Campus property crime in general, and larceny in particular, exhibited similar trends until 1985 when 
the campus rate began to decrease as the national rate increased. Burglary and vehicle theft rates are 
substantially higher in the nation than they are on campuses, but burglary rates have been falling 
while campus vehicle thefts have remained essentially level. It seems logical to attribute the overall 
improvement in campus crime rates to local crime prevention efforts. 

The Correlates of Campus Crim e 

What are the community, organizational, and student characteristics that are most strongly 
associated with campus crime? As noted above, we merged data from the 1990 FBI Uniform Crime 
Report with data from the 1989-90 College Board Survey, as well as data from other sources, in order 
to examine the relationships between crime rates and campus and student characteristics. Table 4 
displays the rates of violent crime, property crime, and total crime by campus type. 
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Table 4 

Campus Crime Rate (Per 100,000) By carnegie Type 

(N=390) 

Type of Crime 



Carnesie Tvpe 


N 


Violent 


Property 


Total 


Two-year 


61 


36 


1507 


1543 


Liberal-arts 11 


7 


77 


2513 


2590 


Comprehensive II 


15 


209 


1778 


1986 


Comprehensive I 


1 CI 


1A 


l ens 

1 7JJ 




Doctoral II 


25 


70 


2028 


2098 


Doctoral I 


31 


67 


2662 


2729 


Research II 


23 


65 


3109 


3174 


Research I 


44 


79 


4075 


4153 


Medical and other 


18 


99 


10744 


10843 



Note: Violent crime includes aggravated assault, armed robbery, forcible rape, and murder. 
Property crime includes larceny, burglary, and vehicle theft. 



The lowest violent and property crime rates are at two-year institutions — campuses that are 
mostly non-residential. The highest rates, especially property crime, are at medical schools and 
health science centers - institutions that are located generally in inner cities with expensive 
equipment and many affluent personnel. The highest rate of violent crime (208 per 100,000) is found 
at Comprehensive II campuses, mostly state colleges in relatively small college towns. The most 
selective Liberal Arts I schools (although there are only three in the sample) are characterized by 
relatively low rates of violent crime (40 per 100,000) and relatively high rates of property crime 
(6723 per 100,000). 

For the 390 institutions in the sample, Charts 3 and 4 compare the campus crime rates with 
the crime rates in the cities and communities in which they are located. Chart 3 shows that students 
are 6 to 1 0 times safer from violent crime when they are on campus than when they are in the 
community. The shading on the bars shows that assault is the most frequent type of violent crime, 
both on campus and off. Two year institutions on average are housed in communities with the lowest 
rates of violent crime, and health sciences centers tend to be located in communities with the highest 
rates. 

Chart 4 shows the corresponding comparison for property crime rates. The crime rates in 
their surrounding communities exceed those on campus for every type of institution, except for 
medical and health institutions, where the average of 10,744 per 100,000 exceeds even tHt of the 
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CHART THREE 



On Campus versus Community Violent Crime 
Rate (Per 100,000) by Carnegie Type 



Two Year 

(N=61) 



Community 
Campus 




Community 

Liberal-arts 

(N=10) Campus 



136 



Community f^^mm m^^ 1 ^ 562 
Comprehensive L_ 
(N=178) Campus -HP 86 



Doctoral 

(N=56) 



Research 

(N=67) 



Community 
Medical & other „ . . 

(N=18) Campus $m" 




Community 
Campus 

Community ^immmSmMS^ 596 
Campus 



0 200 400 600 800 1,000 



Murder&Rape Assault iff Robbery 



9 

ERIC 



278 



317 



CHART FOUR 
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On Campus versus Community Property Crime 
Rate (per 100,000) by Carnegie Type 



Two Year 

(N=61) 



Liberal-Arts 

(N=10) 



Community 
Campus 

Community 
Campus 





• ■.">>- ' 




f. 






1,495 





5,496 



Community 

Comprehensive 

(N=178) Campus 





^•'■■•'■'■■■v.-.y 













Doctoral 

(N=56) 



Research 

(N=67) 



Community 
Campus 

Community 



Campus -\ 



Community 

Medical & Other 



(N=18) 



Campus 



6,170 



3,777 



5,558 



1,922 



6,961 



2,378 



6,598 



m 



] 3,743 







::,:: : : : y.-.-. 




: :x-S>XxX':x--: ; : : :>. 











9,186 



10,705 



Vehicle Theft 



0 2,000 4,000 6,000 8,000 10,000 
Burglary Larceny 



279 



31 



cities in which they are located. [A partial explanation for this finding may be the relatively small 
number and proportion of medical students at these institutions in comparison to the total number of 
employees and visitors. This translates a few crimes into a high rate.] Chart 4 also shows that, 
compared to larceny, the average rates of campus vehicle theft and burglary are small. 

Correlates of Violent Crime 

As noted above, we used the merged database and hierarchical regression to examine the 
relationships among crime rates and campus and student characteristics. Based upon prior theory and 
research, we added the community and structural variables to the regression first. In other words, we 
made the assumption that campuses are more likely to attract crime than to cause it. especially in 
view of the patterns in Charts 3 and 4. Likely offenders are generally present in greater numbers off- 
campus, than on-campus. We also assume that the community variables and the campus 
organizational characteristics are more enduring than individual students and their aggregate 
characteristics. Next we added the set of student characteristics into the hierarchical regression to 
examine their unique contribution to the explained variance. Lastly, we added campus police per 
capita on the assumption that police staffing is a response to crime and not a cause of it. 

Table 5 shows the regression results for violent crime (mostly assault) in the first column, for 
property crime (mostly larceny) in the second column, and for the total crime rate in the third column. 

The first column in Table 5 shows that our community variables by themselves explain an 
insignificant 3% of the variance in violent crime. Even the level of violent crime off-campus bears 
little relationship to violent crime on-campus. The campus organizational measures account for 
another 5% of the variance, also not significant. However, the student variables account for a highly 
significant 27%. Apparently, violent crime is more strongly associated with the nature of the 
students, than with the nature of the campus and the community within which it is located. 

The fourth step in the hierarchical regression in Table 5 adds the level of campus police on 
the grounds that this is most probably an institutional response to crime. This single variable, though 
is not significant and adds only another 6% to the explained variance in violent crime. 

The beta weights in the first column show the results of the final regression with all variables 
in the equation controlling for all others. Beta weights are standardized coefficients - the larger the 
beta, the more influential the variable. The results in Table 5 indicate that campuses with the highest 
rates of violent crime tend to be those with higher than average percentages of minority students, 
campus police, male students, with lower than average cost and selectivity and fraternity life, and 
located in areas with lower than average population and poverty. Acting together, these measures 
explain 41% of the variance. 
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TABLE 5 

RESULTS OF HIERARCHICAL REGRESSION 

(Significant Beta Weights Only) 





Type of Campus Crime 


(Dependent Variable) 
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Total Enrollment 










Number of Full-time Faculty 










Tuition Cost 










Room and Board Cost 


- .34** 








Number of Campus Acres 










Students Per Campus Acre 




-.10 


.10 




Library Holdings per Student 


1 7 








R : Increase 


05(ns) 
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.19** 


— 1 

i 


ST l' DENT CHARACTERISTICS 










Percent from Top 10% of High School 


- .22** 


.62** 


.59** 




Percent in-State 




.25** 


.23** 




Percent Male 


.19 








Percent in Residence Halls 




-.11 






Percent Minority 


.57** 


-.09 






Percent on Financial Aid 




.14* 


.12* 




Percent Fraternity/Sorority 


- .40** 








j R : Increase 


.27** 


.39** 


.36** 


1 


CAMPUS POLICE 


.40** 


.81** 


.83** 




| R* Increase 


.06** 


.21** 


.23** 




| TOTAL R 2 


.41 ♦♦ 


.85** 


.83** 





Note: All beta weights significant at .05 level (ns = non-significant beta weights) 
* significant at .01 
**significant at .001 



Correlates of Property Crime 

The second column in Table 5 shows that our community variables alone explain an 
insignificant 6% of the variance in property crime. Even the level of property crime off-campus bears 
little' statistical relationship to property crime on-campus. On the other hand, the campus 
organizational characteristics account for an additional 19% of the variance in this crime rate, and the 
student variables account for another 39%. Property crime is strongly associated with the nature of 
the campus and its students. 

However, we also find evidence of indirect, rather than direct effects between property crime 
and the nature of the city within which campuses are located. The hierarchical regression results after 
the third step show, not only that the measures of campus selectivity, cost, and resources are 
significantly associated with property crime, but also that population and poverty variables are 
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significant. This finding is roughly consistent also with the results of our stepwise regression (not 
shown here) in which we found that student selectivity, percent in fraternities/sororities, percent 
female, and the population and poverty levels in host cities interacted to account for almost half the 
variance in campus property crime. It appears that relatively affluent and selective campuses are 
more likely to experience property crime, and even more likely still when located in cities with high 
rates of poverty and violent crime. 

The fourth step in the hierarchical regression in Table 5 adds the level of campus police and 
explains another 21% of the variance in property crime. The raw correlation between campus 
property crime and campus police per capita is .76, making the rate of police staffing the best single 
predictor of property crime on campus. 

The beta weights in the property crime column of Table 5 show the results of the final 
regression with all variables in the equation controlling for all others. The most influential variables 
associated with campus property crime are campus police per capita, rank in high school class, 
percent on financial aid, and percent in-state. The student density per acre, percent minority and the 
percent in residence halls are negatively related to property crime. 

The presence of interaction effects is suggested by the fact that four community variables are 
significant in the final equation, even though they failed to prove influential in the first step of the 
hierarchical regression. Average temperature and being located in a heavily populated area are 
negatively related to property crime, while violent crime rates and percent below the poverty level is 
positively related to property crime. When these 10 variables with significant betas interact together, 
they explain 85% of the variance in campus property crime. 



Correlates of Total Crime 



The regression on total campus crime produced results similar to the regression for property 
crime. This was expected because the campus property crime rate in 1990 was 36 times greater than 
the campus violent crime rate across the entire sample. The last column in Table 5 displays an R- 
square pattern consistent with the second column: namely, a lack of significance after the first group 
of community variables are entered, an R-square increase of .19 after the campus measures are 
entered, and an increase of .36 at the third stage when the student characteristics are entered. As was 
true with property crime, there is a very high zero-order correlation between the total campus crime 
rate and campus police per capita (r = .75). 

Before the campus police variable is entered, the third stage of the regression displays a large 
number of significant relationships between total crime and a combination of student characteristics 
(especially student selectivity), and of community traits (especially population and temperature). 
Having a high percent of students on financial aid and a low density of students per campus acre, also 
are related to this crime rate. Again we conclude that these variables interact with each other to 
provide the environment within which campus crime occurs. 

Looking across the three columns of Table 5, there are only three variables that display 
significant beta weights in all three regressions. The area population is consistently negatively related 
to crime. This finding is not congruent with expectations. High student selectivity in terms of rank 
in class is associated with low rates of violent crime but with high rates of property and total crime. 
Other studies suggest that this measure may also be a proxy for student affluence. Finally, campus 
police presence is the strongest indicator of the presence of reported crime. 
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We were interested in the possibility of interaction effects, especially in view of the property 
crime regression results. The community characteristics, in particular, seem to be important only in 
combination with certain student characteristics. To test this hypothesis decided to conduct a 
hierarchical regression analysis entering the campus variables first and the community variables 
afterwards. Table 6 shows the R-square comparisons between the two procedures (community 
measures first versus student characteristics first). In the two cases, the results are not dramatically 
different, but the R-square change for the community variables with property crime does rise from .06 
to . 1 1 under the two different procedures. This tends to confirm the presence of significant 
interaction effects. 



Table 6 






Hierarchical Regression Results for Campus Crime 


Changes in R 2 Values 




Sources of Variance in Crime: 


Violent 


Property 


With Community Variables First: 






Community Variables 


.03 


.06 


Campus Organizational Measures 


.05 


.19* 


Student Characteristics 


.27* 


.49* 


Campus Police 


M 


.11* 


Total R 2 


.41* 


.85* 


With Student Characteristics First: 






Student Characteristics 


.27* 


.43* 


Campus Organizational Measures 


.04 


.10 


Community Variables 


.04 


.11* 


Campus Police 


,06 


.21 * 


Total R 2 


.41* 


.85* 


j * = Significant R 2 change. 



DISCUSSION AND CONCLUSION 



The literature contains few studies on the topic of campus crime, despite its importance. 
This study utilizes three national databases, as well as data from other sources, to examine, first, the 
trends in campus crime, and second, the correlates with various community, organizational, and 
student measures. The study produced several intriguing findings. 

First, despite the impressions one might receive from the media, campus crime rates are 
falling, and they are falling in all categories except vehicle theft which remains level. Moreover, no 
observers believe the decline in campus crime rates can be attributed to declines in the frequency of 
reporting criminal acts by campus victims and police. In fact, the current environment encourages 
the reporting of crime, especially crimes like rape and assault, to a far greater extent than a decade 
ago. 
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The data used for this study does not include all categories of crime, such as weapons 
possession, hate crime, and substance abuse. Beginning in 1993, official reporting requirements will 
be expanded to include these other categories and future researchers will be able to see if particular 
types of crime, not reported here, are more common on college campuses. 

Second, campuses are much safer than the communities in which they are located. The cities 
and counties in which colleges are located generally experience twice the rate of property crime and 
ten times the rate of violent crime than the campuses themselves. In fact, we believe the contrast 
between campus and community crime rates in reality is even more extreme because our data 
overestimate campus crime by including only crime per 100,000 students, ignoring the presence of 
employees and visitors. On many campuses, faculty and staff add another 20% or 30% to the campus 
full-time population, and large sports events like football and basketball attract many thousands of 
visitors to events that are notoriously associated with criminal acts, according to many campus police 
officials. Basing crime rates on the number of students is statistically convenient, but future studies 
should attempt to calculate rates based upon more realistic campus population estimates. 

Third, we find major differences in crime rates at different types of colleges and universities. 
Compared to students at two-year colleges, those in medical schools and health science centers are 3 
times more likely to be victimized by violent crime, and 7 times more likely to experience property 
crime. However, some of these results derive from the small population at some institutions where a 
few crimes translate into a high rate per 100,000 students. 

The Elements of Crime and Crime Spillover 

Criminologists typically consider that three elements must be present in order for crimes to 
occur. First, there must be an offender who is sufficiently motivated, and perhaps skilled enough, to 
commit a crime. Second, there must be a target of the crime — for example, an auto to steal, a person 
to assault, or a stereo to take. Third, the target of the crime must lack a sufficient guardian to deier 
the crime. This simple notion can go a long way in explaining crime on campus. 

Given this model, campuses that have a high percentage of students living in dormitories 
should expect high rates of burglary and larceny. This is because students who live on campus bring 
lots of lightweight durable goods with them (computers, stereos, televisions, and the like). These 
possessions make great targets for theft and burglary. Furthermore, the wealthier the student, tfie 
better the merchandise they will bring to campus. So, campuses with high dormitory populations and 
wealthy students should have even higher burglary and larceny rates. To make matters worse, 
students tend to be young, trusting, and naive. They are lousy guardians of their belongings. Not 
only do young people make good victims, they are also at a prime age to be involved in criminal 
activity. Alternatively, schools with large numbers of commuters and low dormitory populations 
should experience more problems with auto theft. Simply put, they have a lot of automobiles sitting 
around waiting to be stolen. 

However, it is not only the characteristics of the campus and the students that make crime 
possible. Offenders can spill over from the community to the campus. When asked why he robbed 
banks, Willie Sutton replied: "Because that's where the money is." Offenders typically seek the 
highest payoff from a crime for the lowest cost. Certain types of offenders from the community may 
see the campus as being a soft target relative to targets in the community. This should be particularly 
true of economically motivated crimes that require a modicum of criminal expertise. In other words, 
crimes like motor vehicle theft, burglary, and robbery on campus may draw offenders from the 
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community. The type of crime perpetrated depends upon the exact combination of campus, student, 
and community characteristics. 

It certainly appears from our data that different types of crime exhibit different dynamics and 
patterns of causality. Our combinations of community variables, organizational measures, and 
student characteristics are better at accounting for campus property crime than violent crime. While it 
is more difficult to identify the variables that are associated with violent crime, our findings regarding 
campus property crime are relatively consistent with Routine Activities Theory and the Willie Sutton 
remark. Before the heavily influential campus police variable was entered into the regression, our 
analysis showed that 64% of campus property crime was associated with selective and affluent 
campuses located in cities and counties with high rates of poverty. 

Most of the property crime rate consists of larceny -an offense that does not usually require 
professional talent. The other components of armed robbery and auto theft, on the other hand, are 
most often conducted by professional offenders. College students themselves may carryout larceny, 
but they are unlikely to work their way through college by means of armed robbery and auto theft. 
This suggests a separate study aimed at specific types of campus crime because the dynamics are very 
different. 

The evidence for spillover from community to campus in this study is statistically significant, 
but the evidence is not consistent. While we found evidence of a spillover effect for property crime, 
we did not for violent crime. We successfully accounted for 85% of the variance in campus property 
crime, but only 41% of the variance in campus violent crime. What we did find out about violent 
crime, unfortunately, is consistent with national trends. Violent crime, three-fourths of which is 
assault, is more prevalent at campuses with above average percentages of minority students, campus 
police, male students, and below average cost and quality. This is no doubt a source of concern for 
all in higher education. 

Our study obtained some results that were unexpected. While national crime rates are 
associated with urbanness, campus crime is not necessarily. For example, campus density in 
students per acre is negatively associated with property and total crime. Apparently, the more 
students are spread out, the more hospitable are the circumstances for property crime to occur. 
Another finding that is inconsistent with national crime patterns is that average temperature is 
negatively related to property and total crime. Northern locales generally experience less crime, but 
northern campuses in our study experienced more property crime. 

We are not sure how to interpret the high relationship between campus crime and campus 
police. The high presence of campus police on campuses where crime is occurring may be a sign that 
administrators are acting responsively, or it may mean that crimes are more often reported and 
officially recorded on such campuses. The reporting issue is of particular interest to institutional 
researchers because they are the data managers and questionnaire respondents on most campuses. 
Studies like ours are heavily dependent upon accurate reporting. To the extent that victims, 
campuses, and localities under report crime, it interferes with our ability to understand its causality 
and to develop appropriate policy responses. 
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